bpwwer Posted January 24, 2020 Author Share Posted January 24, 2020 7 hours ago, f11 said: Sorry, I'm a git-idiot - it took me a while (well, quite a while) to find your modifed 'post' version of weewx-crt - I didn't realize it was under Branches. BTW, the crt.py preamble comments at the top shows both the original 'filename = /path/to/realtime.txt' and a newer 'realtime_txt = /path/to/realtime.txt under two different references to [CumulusRealTime] in weewx.conf. I already had the original reference, but added the new one just in case. I suspect both are probably not needed. Sorry, I should have mentioned that it was on the branch. I tend to use branches when making changes to something to keep those changes separate from the original. I think crt.py supports both methods to specify realtime.txt. Looks like it first tries the option 'filename' and if that doesn't exist, it tries 'realtime_txt'. 5 hours ago, f11 said: WeatherPoly is doing its thing, proved by issuing curl POST request from command line - updates arrive at Polisy as expected and are decoded properly. One thing: if a parameter is '---' in the realtime.txt file, it causes problems for the conversion to float (for wind driver. at least): 2020-01-24 04:25:08,251 [Thread-1 ] [INFO ] - Set wind driver GV0 to --- 2020-01-24 04:25:08,252 [Thread-1 ] [DEBUG] - setDriver failed 11 -> wind could not convert string to float: '---' The data that led to that error was wind-winddir : field 11 (right after the pressure reading 1007.2): pi@RPi3-3-Weewx:/etc/weewx/skins/Seasons $ cat /var/tmp/realtime.txt 24-01-20 04:25:00 -6.2 87 -8.0 0.0 0.0 NULL 0.0 0.0 1007.2 --- 0 m/s C mb mm 0.0 -0.5 0.2 0.2 0.0 20.0 35 -6.2 -2.2 -3.3 00:55 -6.2 04:25 0.0 00:05 1.3 00:35 1007.8 01:40 1007.1 00:25 3.9.2 0 0.0 -6.2 -6.2 NULL 0.0 0 NULL 0.0 0 0 0 --- 926 m -9.1 0.0 NULL 0 I'm probably not getting auto-updates due to fumbling the installation of the modified crt.py file. I'll have to check it in the morning. You should be able to copy the crt.py over the existing one. On my system it is located in /usr/share/weewx/user. In the weewx config file you did add the 'realtime_url = http://<ip>:8080/weewx' right? That's what tells crt.py to post the data. You can also check the weewx log and it should report an error if the post request failed. It will also send info to the debug log for some errors, but I'm not sure how to enable the debug logging. My weewx log is going to /var/messages I believe. Link to comment Share on other sites More sharing options...
f11 Posted January 24, 2020 Share Posted January 24, 2020 (edited) Got it figured. Didn't know I could use wee_extension installer with a directory of the unpacked weewx-crt files - amazing (!) what you can learn when you read the python code itself Huh ... no updates. Obviously I've forgotten SOMETHING. I just don't know what. [ edit: our messages crossed in posting - just checking my weewx.conf file now ... [Engine] [[Services]] . . . . . . archive_services = weewx.engine.StdArchive, user.crt.CumulusRealTime . . . [CumulusRealTime] filename = /var/tmp/realtime.txt realtime_txt = /var/tmp/realtime.txt # realtime_xml = /var/tmp/realtime.xml realtime_url = http://192.168.1.23:8080/weewx # update file on archive period binding = archive # specify my preferred date stamp field separator date_separator = - # units preferences for realtime.txt values unit_system = METRIC wind_units = meter_per_second temperature_units = degree_C pressure_units = mbar cloudbase_units = meter rain_units = mm Maybe I'm not seeing something obvious? I used ''sudo service weewx stop', waiting a minute, then 'sudo service weewx start'. The status option looked normal. Both the Polisy Log and the WP Log show zero activity from WP after starting. Checking /var/log/messages for weewx and crt statements ... hmm, after weewx and crt are up and running, the log only shows weewx general activity, but crt is never heard from again. I'm going to add print statements into crt.py to see what's going on ... this is TOO strange. Hold on ... just scanning through the 'crt.py' file in /usr/share/weewx/user ... it doesn;t have your changes to class CumulusRealTime(StdService) that I was expecting (I can't see references to the self.realtime_url and other things from your earlier patch post). Could have SWORN I dropped your new crt.py here. So, this must be where I dropped the ball. Give me a few minutes to see if I can unwind this snafu. Edited January 24, 2020 by f11 Link to comment Share on other sites More sharing options...
bpwwer Posted January 24, 2020 Author Share Posted January 24, 2020 I'm seeing this in the log (/var/log/messages) when it starts Jan 24 10:59:12 piserver weewx[17156]: crt: service version is 0.20 Jan 24 10:59:13 piserver weewx[17156]: crt: realtime txt output goes to /var/tmp/realtime.txt Jan 24 10:59:13 piserver weewx[17156]: crt: realtime txt output posted to http://192.168.92.11:8080/weewx You could add a log output to the post_data function in crt.py so that it outputs something every time it tries to send the data. Maybe something like. def post_data(self, url, data): try: r = requests.post(url = url, data = data) if r.status_code != 200: logerr("crt: Failed to post realtime data to %s: %d" % (url, r.status_code) else: loginf("crt: Successfuly posted realtime data to %s" % url) except Exception, e: logdbg("crt: Exception while sending realtime data to server.") Then I see messages like: Jan 24 11:27:27 piserver weewx[19639]: crt: crt: Sucessfuly posted realtime data to http://192.168.92.11:8080/weewx Link to comment Share on other sites More sharing options...
f11 Posted January 24, 2020 Share Posted January 24, 2020 (edited) My weewx is running as root, so I'm used to using 'sudo' a LOT ... and sometimes forget that some programs are supposed to run as a regular user. I assume that weewx-crt should run as root as well? Just checking for permissions issues. -> Edit: ok, think I know what happened - at 4AM when I last downloaded from your repo, I grabbed the master files, NOT the 'post' branch, although I knew I needed the post files ... why I do these things at that time of day I have NO idea - yup, a git-idiot. After that I did indeed install everything but they're Matt's crt, not yours. I guess that might explain the missing log messages ??. I'll fix this up and report back. -> Ah-ha - I've been using the green 'Clone or Download' button and copying the url line from the drop down - which just downloads the master files. Trying to figure out how to download the branch files. Edited January 24, 2020 by f11 Link to comment Share on other sites More sharing options...
mwester Posted January 24, 2020 Share Posted January 24, 2020 8 minutes ago, f11 said: ... and sometimes forget that some programs are supposed to run as a regular user... As a general rule, NO programs should run as root -- but specifically as they relate to node servers, you can't run as root on the Polisy, which might be an issue if the code in question is to run there (no reason that weewx couldn't run on the Polisy -- that's certainly a tool I'd like to move off the RPI 'cause I feel a lot better about a proper SSD compared to the reliability of an SD card). Just an FYI for node server and Polisy developers. 1 Link to comment Share on other sites More sharing options...
f11 Posted January 24, 2020 Share Posted January 24, 2020 (edited) Yup, I understand the follies of running as root - but when I installed weewx way back when, that's how it got installed and I've been reluctant to make the necesary changes. I usually install into my user folders. Yeah, Polisy requires nodeservers to run as polyglot which is what I've got going. I installed an m-sata 250GB SSD on my weewx RPi over a 2 years ago, worrying about the wear and tear on the micro-SD card with all the file writing and re-writing. So everything but the /boot files are on the SSD. pi@RPi3-3-Weewx:~/crt $ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/root 239314556 7323308 219764996 4% / devtmpfs 467512 0 467512 0% /dev tmpfs 472120 0 472120 0% /dev/shm tmpfs 472120 47700 424420 11% /run tmpfs 5120 8 5112 1% /run/lock tmpfs 472120 0 472120 0% /sys/fs/cgroup /dev/mmcblk0p6 67434 24154 43280 36% /boot tmpfs 94424 0 94424 0% /run/user/1000 Edited January 24, 2020 by f11 Link to comment Share on other sites More sharing options...
f11 Posted January 24, 2020 Share Posted January 24, 2020 (edited) Downloaded the .zip file from post, copied your crt.py over Matt's, restarting weewx and waiting ... and messages, updates, and all the rest !! Here's the WP log (hmm, I got these lines twice ... anyway) : 2020-01-24 13:15:22,190 [Thread-1 ] [DEBUG] Got some WeeWX data 2020-01-24 13:15:22,192 [Thread-1 ] [INFO ] - Set temperature driver ST to 4.3 2020-01-24 13:15:22,193 [Thread-1 ] [INFO ] Updating Driver temperature - ST: 4.3, uom: 4 2020-01-24 13:15:22,195 [Thread-1 ] [INFO ] - Set temperature driver GV0 to -0.9 2020-01-24 13:15:22,196 [Thread-1 ] [INFO ] Updating Driver temperature - GV0: -0.9, uom: 4 2020-01-24 13:15:22,198 [Thread-1 ] [INFO ] - Set humidity driver ST to 69 2020-01-24 13:15:22,200 [Thread-1 ] [INFO ] Updating Driver humidity - ST: 69.0, uom: 51 2020-01-24 13:15:22,202 [Thread-1 ] [INFO ] - Set temperature driver GV1 to 4.3 2020-01-24 13:15:22,202 [Thread-1 ] [INFO ] Updating Driver temperature - GV1: 4.3, uom: 4 2020-01-24 13:15:22,204 [Thread-1 ] [INFO ] - Set light driver GV0 to 109 2020-01-24 13:15:22,205 [Thread-1 ] [INFO ] Updating Driver light - GV0: 109.0, uom: 74 2020-01-24 13:15:22,207 [Thread-1 ] [INFO ] - Set temperature driver GV4 to 20.8 2020-01-24 13:15:22,207 [Thread-1 ] [INFO ] Updating Driver temperature - GV4: 20.8, uom: 4 2020-01-24 13:15:22,209 [Thread-1 ] [INFO ] - Set humidity driver GV0 to 34 2020-01-24 13:15:22,210 [Thread-1 ] [INFO ] Updating Driver humidity - GV0: 34.0, uom: 51 2020-01-24 13:15:22,212 [Thread-1 ] [INFO ] - Set pressure driver GV1 to -2.0 2020-01-24 13:15:22,213 [Thread-1 ] [INFO ] Updating Driver pressure - GV1: -2.0, uom: 25 2020-01-24 13:15:22,215 [Thread-1 ] [INFO ] - Set pressure driver ST to 1003.2 2020-01-24 13:15:22,216 [Thread-1 ] [INFO ] Updating Driver pressure - ST: 1003.2, uom: 117 2020-01-24 13:15:22,218 [Thread-1 ] [INFO ] - Set wind driver GV4 to 0.4 2020-01-24 13:15:22,220 [Thread-1 ] [INFO ] Updating Driver wind - GV4: 0.4, uom: 32 2020-01-24 13:15:22,222 [Thread-1 ] [INFO ] - Set wind driver GV1 to 0.4 2020-01-24 13:15:22,224 [Thread-1 ] [INFO ] Updating Driver wind - GV1: 0.4, uom: 32 2020-01-24 13:15:22,226 [Thread-1 ] [INFO ] - Set wind driver GV0 to SE 2020-01-24 13:15:22,227 [Thread-1 ] [DEBUG] - setDriver failed 11 -> wind could not convert string to float: 'SE' 2020-01-24 13:15:22,229 [Thread-1 ] [INFO ] "POST /weewx HTTP/1.1" 200 - I may have the failed 'wind' issue because I grabbed the wrong value - I see I grabbed the ordinal wind dir, not the bearing. Edited January 24, 2020 by f11 Link to comment Share on other sites More sharing options...
f11 Posted January 24, 2020 Share Posted January 24, 2020 Yup, consistently getting two sets of identical log file entries for each update, varying only by time stamp in the msec field. Also, wind direction, both ordinal and bearing, are having problems converting ordinals (SE, WSW, - - - , etc) and bearings=NULL / - - - into floats, understandably. 2020-01-24 14:20:21,995 [Thread-1 ] [INFO ] - Set wind driver GV0 to NULL 2020-01-24 14:20:21,995 [Thread-1 ] [DEBUG] - setDriver failed 7 -> wind could not convert string to float: 'NULL' Link to comment Share on other sites More sharing options...
bpwwer Posted January 24, 2020 Author Share Posted January 24, 2020 4 minutes ago, f11 said: Yup, consistently getting two sets of identical log file entries for each update, varying only by time stamp in the msec field. Also, wind direction, both ordinal and bearing, are having problems converting ordinals (SE, WSW, - - - , etc) and bearings=NULL / - - - into floats, understandably. 2020-01-24 14:20:21,995 [Thread-1 ] [INFO ] - Set wind driver GV0 to NULL 2020-01-24 14:20:21,995 [Thread-1 ] [DEBUG] - setDriver failed 7 -> wind could not convert string to float: 'NULL' I'm not sure I understand what you mean by two sets of identical log entries. Do mean the whole block between "Got some WeeWx data" and "POST /weewx HTTP/1.1 200" is duplicated? That would imply that you are actually getting two requests from WeeWx (or have two copies running maybe). Or are you referring to the 1 hour ago, f11 said: 2020-01-24 13:15:22,192 [Thread-1 ] [INFO ] - Set temperature driver ST to 4.3 2020-01-24 13:15:22,193 [Thread-1 ] [INFO ] Updating Driver temperature - ST: 4.3, uom: 4 two lines above? Those are normal. The first is my code saying it's telling Polyglot to send an update to the ISY and the second is Polyglot saying it is sending the up to the ISY. I tend to sometimes over log things. I've started converting my node servers over to have user configurable logging along with some other improvements but I've not yet cleaned up this node server. Link to comment Share on other sites More sharing options...
f11 Posted January 24, 2020 Share Posted January 24, 2020 Yes, 'the whole block between "Got some WeeWx data" and "POST /weewx HTTP/1.1 200" is duplicated'. The two blocks are identical in all regards, except for their timestamps, which only differ in msec. Checked for multiple instances of weewx using 'ps aux' and 'htop' -> nope, just one that's visible ... checking for multiple crt extensions in weewx using wee_extensions --list -> nope, just one is listed. Did an update and dist-upgrade of the weewx RPi just in case - it goes long periods without restarting or being updated) - then rebooted Polisy, and rebooted ISY so everything is starting out in a somewhat known state. Everything works like a damn - except for the two blocks of identical log entries. Weird. Is there a way to temporarily disable weewx-crt without stopping-starting everything? If I have some kind of double-instance action going on, maybe I can shut down the one I can see to help me find the one I can't. Since you're not seeing this double-log effect, its obviously at my end. I'll keep looking. But otherwise, everything looks great with WeatherPoly and your mod'd version of weewx-crt : updates come in with only slight latency from the update of realtime.txt to arrival in WP's log, and the data transfers across to ISY smoothly and appears on the ISY screen as expected. The only issue I've seen so far is the failed conversion of realtime.txt 'none/NULL/- - -/N NE E SW etc' strings to values in WP, clearly an artifact of dealing with the incoming file fields instead of live data. Although I've only seen it show up in wind direction (both ordinal and bearing) custom parameter fields, its likely to be common to all converted fields. I was going to test that next, just assigning random fields to various custom parameters to see if WP can handle unexpected inputs gracefully. Link to comment Share on other sites More sharing options...
bpwwer Posted January 25, 2020 Author Share Posted January 25, 2020 I'm not seeing the duplicates. I do get updates every few seconds but the messages from WeeWX in the /var/log/messages correspond to the log entries in the WeatherPoly log. Currently, WeatherPoly handles bad data by reporting the error and ignoring it. Anything that's not numeric is considered bad data. Link to comment Share on other sites More sharing options...
f11 Posted January 25, 2020 Share Posted January 25, 2020 Ah - you've set your system to update on 'loop' (~3 sec period) while mine is set to update on 'archive' (300 sec period). Ok, so the WP log message regarding wind direction isn't really an 'error', its just flagging what it considers bad data. In that case, I've run across zero issues so far. weewx-crt starts and stops gracefully in tandem with weewx (which I would expect), and other than logging activity just hums along in the background. So far, things are working extremely well. Link to comment Share on other sites More sharing options...
f11 Posted January 25, 2020 Share Posted January 25, 2020 Think I found a culprit in weewx.conf: [Engine] [[Services]] . . . process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate, user.crt.CumulusRealTime archive_services = weewx.engine.StdArchive, user.crt.CumulusRealTime . . . 'user.crt.CumulusRealTime' is listed at the end of both 'process_services' and 'archive_services'. Obviously this is my doing, but not sure if I did it for a reason, or just fat-fingered the entry. Looking back in my archives, this doubling of service entries first shows up in my 24Jan 11:36AM update of weewx.conf. By that time, I was working with a lot of files and folders, so I'm guessing it was a fat-finger snafu. Doesn't matter. I fixed the [Engine] entries, stopped and re-started weewx, and watched the logs -> ??SUCCESS ??!! The next update showed up in the WP log (with what seemed to be a lot less latency) as only one block of entries. Glad that's sorted! THANKS so much for your help with this !! I'd have NEVER sorted all this out on my own. I also appreciate the opportunity to tag along with a professional coder, I learned a lot. I'll keep poking at WP and weewx-crt and let you know what I find. Best Regards, Rod Link to comment Share on other sites More sharing options...
f11 Posted January 29, 2020 Share Posted January 29, 2020 Bob, just an update ... I've been poking at WeatherPoly and your weewx-crt mod'd code, trying to break things, for the past several days. Its really robust, and so far seems impervious to unexpected data or mis-configurations. Its been running without interruption now since last Friday, Jan 24th, and hasn't interfered with weewx in any way that I've seen. I'm presently transferring 3 wind, 2 pressure, 2 humidity, 1 light (radiation sensor), and 4 temperature values from my Davis Vantage Pro 2 station to WeatherPoly via the weewx-crt mod'd code - all have been updating 24/7 since Friday without issue. Same report for WeatherPoly regarding its modifications to handle the auto-sent weewx-crt file. Everything just works, even if I reboot the RPi running weewx, bounce the power on Polisy, or interrupt the data coming from the weather station. The only comment I'd add is that I look forward to being able to reduce WP log entries via configurable WP setting, as you said you were considering. Now that everything is running smoothly, the number of logged entries can be cut back considerably, in both the Polisy and WP logs. If there's any specific kind of testing you want me to do on my system, let me know. Otherwise, I'm a pretty happy camper ! Thank you ! Rod Link to comment Share on other sites More sharing options...
stre1026 Posted February 16, 2020 Share Posted February 16, 2020 I'm very glad I found this post about your efforts with WeeWx because I was just about to go down that rabit hole with my system. I have WeeWx running with a DIY weather station and I'm currently uploading to PWSweather and Wunderground. However, with the climate module being discontinued, I am looking for alternatives. I was going to have WeeWx output JSON via the method I found here: https://hoke.org/john/2018/05/using-weewx-for-home-automation/ and try to write a node server to parse it, but it seems like you gentlemen found a solution that works. I will see if I can follow your progress posts and replicate what you have done on my system. Thanks! Steve 1 Link to comment Share on other sites More sharing options...
stre1026 Posted February 16, 2020 Share Posted February 16, 2020 Just an update to my reply above. I was able to get this working on my ISY/Polisy as well! Thank you gentlemen for your great work on this. Saved me a ton of time! Link to comment Share on other sites More sharing options...
bpwwer Posted February 16, 2020 Author Share Posted February 16, 2020 And to update the status of the modified weewx-crt extension. Matt rejected the pull request to add this ability to his extension and he had some pretty good reasons. Mainly he thought it was a bit outside the scope of his extension and would rather see it as a separate extension. He also noted that as it's currently written, a problem with the HTTP request could block WeeWx for an extended amount of time which is not a good thing. I understand his concerns and agree that the it should be fixed so that it can't block WeeWX. However, I'm not really using WeeWX and don't really want to try and maintain an extension for it. If someone who is willing to maintain an extension wants to step up, I'd be happy help modify my version of the weewx-crt so that it can be a standalone extension. In the meantime, since this is working, I'll keep my fork up there on github for anyone that needs it. Link to comment Share on other sites More sharing options...
bigDvette Posted February 23, 2020 Share Posted February 23, 2020 Not sure if I missed instructions somewhere, but finally figured out I had to map the nodes form this list: https://github.com/bpaauwe/WeatherPoly to positions in the url. I can't see the url coming in so I looked it up in CRT.py. I added your changes to my CRT.py file that I use to publish data for a local weather page using the realtime.txt. I can create the variables for yesterday records you have commented out if anyone would like them. I'm mostly writing this to document what I had to do to get it working. If mapping temperature-main = 2 was incorrect then I'd like to know the right method. Also, in the CRT.py temp is line 3 but the first 2 or dates so trail and error got me there. I have a request. I actually pull in some other weather data through some extensions to weewx. In particular I pull in the lake level from the geological site. Could we add node category and maybe call it environment. I'd like variables for lakelevel (current, max_year, min_year, max_month, min_month, min_day, max_day) . It could just be water level so it is more generic. All these are in feet of elevation so convert from meter to feet ... I could see other variables like snow in this category. If I still had polyglot on my raspberry instead of the polisy I'd hack the files to see I could get it working. Link to comment Share on other sites More sharing options...
bpwwer Posted February 24, 2020 Author Share Posted February 24, 2020 18 hours ago, bigDvette said: Not sure if I missed instructions somewhere, but finally figured out I had to map the nodes form this list: https://github.com/bpaauwe/WeatherPoly to positions in the url. I can't see the url coming in so I looked it up in CRT.py. I added your changes to my CRT.py file that I use to publish data for a local weather page using the realtime.txt. I can create the variables for yesterday records you have commented out if anyone would like them. I'm mostly writing this to document what I had to do to get it working. If mapping temperature-main = 2 was incorrect then I'd like to know the right method. Also, in the CRT.py temp is line 3 but the first 2 or dates so trail and error got me there. I have a request. I actually pull in some other weather data through some extensions to weewx. In particular I pull in the lake level from the geological site. Could we add node category and maybe call it environment. I'd like variables for lakelevel (current, max_year, min_year, max_month, min_month, min_day, max_day) . It could just be water level so it is more generic. All these are in feet of elevation so convert from meter to feet ... I could see other variables like snow in this category. If I still had polyglot on my raspberry instead of the polisy I'd hack the files to see I could get it working. The indexing in WeatherPoly is zero based while the list in crt.py starts at one. So the index is 1 less than the line #. That probably isn't documented anywhere. While it's possible to add more node types and more data values, it does take a bit of effort and because the way Polyglot and the ISY are currently architected, everything tends to be fairly static. What that means is that adding in something like lake level is probably not something that would be used by very many people (I.E. probably just you at this time). If there was a way to do additions like this with just changes to the configuration, I'd do that. But I currently don't have the time to come up with a way to add user defined nodes/values. BTW, it's just about as easy to hack on the files in a Polisy as it is in a RPi. Link to comment Share on other sites More sharing options...
bigDvette Posted February 25, 2020 Share Posted February 25, 2020 wow, this thing does a lot of info logging that I can't turn off. My polisy becomes unresponsive when running WeatherPoly. My guess is that my crt.py is bound to LOOP and not archive. I ended up copying CRT.py and modifying the class and section so I could have 2 instances running (1 bound to loop for my realtime weather station and 1 bound to archive for ISY). not sure if slowness was due to the logging or the actual processing of variables. Link to comment Share on other sites More sharing options...
bpwwer Posted February 26, 2020 Author Share Posted February 26, 2020 I didn't think it was doing that much logging. Some of my other node servers do a lot more. I do have the goal to add configurable logging to all of them eventually so I added it to WeatherPoly and pushed version 0.1.9 I would think that it would take a lot of processing to make the Polisy become unresponsive. How often does loop execute? I could see if it's once a second and you're sending a bunch of values. Looks like I do have it always send an update even if the value hasn't changed so that's something I could optimize. Link to comment Share on other sites More sharing options...
jca001 Posted February 26, 2020 Share Posted February 26, 2020 (edited) I think not sending values unless they change is a good thing in general. But values should be sent every so often even if they do not change in case ISY is rebooted so the values will get updated after some amount of time. Maybe every 5 or 10 minutes or a configurable parameter. While I am making suggestions I don't think temperatures should be NNN.NN. I think they should be NNN.N. Who needs to know if the temperature changed by 100th of a degree? This can cause programs to trigger every time the value is updated because it changes between say 81.11 and 81.12. Edited February 26, 2020 by jca001 Link to comment Share on other sites More sharing options...
bigDvette Posted February 26, 2020 Share Posted February 26, 2020 Davis weather stations have. 2 second loop packet so it was sending very variable 12, every second. I commented out all the logger.info and it still streamed a lot. I think it is a polyglot setting potentially as harmony also has a tendency to be chatty. i had asked the question if you wanted help with the custom crt.py in terms of getting other variables. I store my lake level in weewx and pull it out and do math on the variables so I’ve figured out the sql methods. i noticed you I had some comments in the file to come back later. Link to comment Share on other sites More sharing options...
bpwwer Posted February 26, 2020 Author Share Posted February 26, 2020 15 hours ago, jca001 said: I think not sending values unless they change is a good thing in general. But values should be sent every so often even if they do not change in case ISY is rebooted so the values will get updated after some amount of time. Maybe every 5 or 10 minutes or a configurable parameter. While I am making suggestions I don't think temperatures should be NNN.NN. I think they should be NNN.N. Who needs to know if the temperature changed by 100th of a degree? This can cause programs to trigger every time the value is updated because it changes between say 81.11 and 81.12. Are you commenting about the same node server as above, WeatherPoly? I just looked at it and all temperature values are rounded to 1 decimal place and the node definition specifies temperature with a precision of 1 decimal place. It should not be reporting temperatures to the 100th of a degree. Link to comment Share on other sites More sharing options...
bpwwer Posted February 26, 2020 Author Share Posted February 26, 2020 14 hours ago, bigDvette said: Davis weather stations have. 2 second loop packet so it was sending very variable 12, every second. I commented out all the logger.info and it still streamed a lot. I think it is a polyglot setting potentially as harmony also has a tendency to be chatty. i had asked the question if you wanted help with the custom crt.py in terms of getting other variables. I store my lake level in weewx and pull it out and do math on the variables so I’ve figured out the sql methods. i noticed you I had some comments in the file to come back later. Doing something every second or two could be the issue. I did just push a new version with configurable logging so that may help some, but it's probably the http rest calls to the ISY that are really the issue. There isn't currently a way to batch them up so every node value update is a separate http request and doing many every second is probably not good for either the Polisy or the ISY. So I'm not really maintaining a custom crt.py as I don't use WeeWX and don't want to try an maintain an extension for something I don't use. The author of the crt.py extension did look at the changes and had a few issues with them. 1) he didn't think they really fit with the goals he had for the extension and suggested it be a separate extension. 2) it shouldn't be doing the http requests from inside the loop. His suggestion was to spawn a thread to do the http request, that way it wouldn't block other loop processing if it takes a while or times out. Moving it archive instead of loop is probably a good idea too. If you're willing to maintain a WeeWX extension for this, I am willing to make any changes necessary to WeatherPoly to continue to work with it. Link to comment Share on other sites More sharing options...
Recommended Posts