Jump to content

Polyglot weather station node servers


bpwwer

Recommended Posts

7 hours ago, f11 said:

Sorry, I'm a git-idiot - it took me a while (well, quite a while) to find your modifed 'post' version of weewx-crt - I didn't realize it was under Branches.

BTW, the crt.py preamble comments at the top shows both the original 'filename = /path/to/realtime.txt' and a newer 'realtime_txt = /path/to/realtime.txt under two different references to [CumulusRealTime] in weewx.conf.  I already had the original reference, but added the new one just in case.  I suspect both are probably not needed.

Sorry, I should have mentioned that it was on the branch. I tend to use branches when making changes to something to keep those changes separate from the original.  

I think crt.py supports both methods to specify realtime.txt.  Looks like it first tries the option 'filename' and if that doesn't exist, it tries 'realtime_txt'.

5 hours ago, f11 said:

WeatherPoly is doing its thing, proved by issuing curl POST request from command line - updates arrive at Polisy as expected and are decoded properly.  One thing: if a parameter is '---' in the realtime.txt file, it causes problems for the conversion to float (for wind driver. at least):


2020-01-24 04:25:08,251 [Thread-1  ] [INFO ]  - Set wind driver GV0 to ---
2020-01-24 04:25:08,252 [Thread-1  ] [DEBUG]   - setDriver failed 11  -> wind could not convert string to float: '---'

The data that led to that error was wind-winddir : field 11 (right after the pressure reading 1007.2):


pi@RPi3-3-Weewx:/etc/weewx/skins/Seasons $ cat /var/tmp/realtime.txt
24-01-20 04:25:00 -6.2 87 -8.0 0.0 0.0 NULL 0.0 0.0 1007.2 --- 0 m/s C mb mm 0.0 -0.5 0.2 0.2 0.0 20.0 35 -6.2 -2.2 -3.3 00:55 -6.2 04:25 0.0 00:05 1.3 00:35 1007.8 01:40 1007.1 00:25 3.9.2 0 0.0 -6.2 -6.2 NULL 0.0 0 NULL 0.0 0 0 0 --- 926 m -9.1 0.0 NULL 0

I'm probably not getting auto-updates due to fumbling the installation of the modified crt.py file.  I'll have to check it in the morning.

You should be able to copy the crt.py over the existing one. On my system it is located in /usr/share/weewx/user.

In the weewx config file you did add the 'realtime_url = http://<ip>:8080/weewx'   right?  That's what tells crt.py to post the data.  You can also check the weewx log and it should report an error if the post request failed.  It will also send info to the debug log for some errors, but I'm not sure how to enable the debug logging. My weewx log is going to /var/messages I believe.

Link to comment
Share on other sites

Got it figured.  Didn't know I could use wee_extension installer with a directory of the unpacked weewx-crt files - amazing (!) what you can learn when you read the python code itself :-)

Huh ... no updates.  Obviously I've forgotten SOMETHING.  I just don't know what.

[ edit: our messages crossed in posting - just checking my weewx.conf file now ... 

[Engine]
    [[Services]]
        . . .
        . . .
        archive_services = weewx.engine.StdArchive, user.crt.CumulusRealTime
        . . .


[CumulusRealTime]
    filename = /var/tmp/realtime.txt
    realtime_txt = /var/tmp/realtime.txt
    # realtime_xml = /var/tmp/realtime.xml
    realtime_url = http://192.168.1.23:8080/weewx
    # update file on archive period
    binding = archive
    # specify my preferred date stamp field separator
    date_separator = -
    # units preferences for realtime.txt values
    unit_system = METRIC
    wind_units = meter_per_second
    temperature_units = degree_C
    pressure_units = mbar
    cloudbase_units = meter
    rain_units = mm

Maybe I'm not seeing something obvious?  I used ''sudo service weewx stop', waiting a minute, then 'sudo service weewx start'.  The status option looked normal.  Both the Polisy Log and the WP Log show zero activity from WP after starting.

Checking /var/log/messages for weewx and crt statements ... hmm, after weewx and crt are up and running, the log only shows weewx general activity, but crt is never heard from again.  I'm going to add print statements into crt.py to see what's  going on ... this is TOO strange.

Hold on ... just scanning through the 'crt.py' file in /usr/share/weewx/user ... it doesn;t have your changes to class CumulusRealTime(StdService) that I was expecting (I can't see references to the self.realtime_url and other things from your earlier patch post).  Could have SWORN I dropped your new crt.py here.  So, this must be where I dropped the ball.  Give me a few minutes to see if I can unwind this snafu.

Edited by f11
Link to comment
Share on other sites

I'm seeing this in the log (/var/log/messages) when it starts


Jan 24 10:59:12 piserver weewx[17156]: crt: service version is 0.20
Jan 24 10:59:13 piserver weewx[17156]: crt: realtime txt output goes to /var/tmp/realtime.txt
Jan 24 10:59:13 piserver weewx[17156]: crt: realtime txt output posted to http://192.168.92.11:8080/weewx

You could add a log output to the post_data function in crt.py so that it outputs something every time it tries to send the data. Maybe something like.

def post_data(self, url, data):
        try:
            r = requests.post(url = url, data = data)
            if r.status_code != 200:
                logerr("crt: Failed to post realtime data to %s: %d" % (url, r.status_code)
            else:
                loginf("crt: Successfuly posted realtime data to %s" % url)
        except Exception, e:
            logdbg("crt: Exception while sending realtime data to server.")

Then I see messages like:

Jan 24 11:27:27 piserver weewx[19639]: crt: crt: Sucessfuly posted realtime data to http://192.168.92.11:8080/weewx

 

Link to comment
Share on other sites

My weewx is running as root, so I'm used to using 'sudo' a LOT ... and sometimes forget that some programs are supposed to run as a regular user.  I assume that weewx-crt should run as root as well?  Just checking for permissions issues.

-> Edit: ok, think I know what happened - at  4AM when I last downloaded from your repo, I grabbed the master files, NOT the 'post' branch, although I knew I needed the post files ... why I do these things at that time of day I have NO idea - yup, a git-idiot.  After that I did indeed install everything but they're Matt's crt, not yours.  I guess that might explain the missing log messages ??.  I'll fix this up and report back.

-> Ah-ha - I've been using the green 'Clone or Download' button and copying the url line from the drop down - which just downloads the master files.  Trying to figure out how to download the branch files.

Edited by f11
Link to comment
Share on other sites

8 minutes ago, f11 said:

... and sometimes forget that some programs are supposed to run as a regular user...

As a general rule, NO programs should run as root -- but specifically as they relate to node servers, you can't run as root on the Polisy, which might be an issue if the code in question is to run there (no reason that weewx couldn't run on the Polisy -- that's certainly a tool I'd like to move off the RPI 'cause I feel a lot better about a proper SSD compared to the reliability of an SD card).  Just an FYI for node server and Polisy developers.

  • Like 1
Link to comment
Share on other sites

Yup, I understand the follies of running as root - but when I installed weewx way back when, that's how it got installed and I've been reluctant to make the necesary changes.  I usually install into my user folders.  Yeah, Polisy requires nodeservers to run as polyglot which is what I've got going.

I installed an m-sata 250GB SSD on my weewx RPi over a 2 years ago, worrying about the wear and tear on the micro-SD card with all the file writing and re-writing.  So everything but the /boot files are on the SSD.

pi@RPi3-3-Weewx:~/crt $ df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root      239314556 7323308 219764996   4% /
devtmpfs          467512       0    467512   0% /dev
tmpfs             472120       0    472120   0% /dev/shm
tmpfs             472120   47700    424420  11% /run
tmpfs               5120       8      5112   1% /run/lock
tmpfs             472120       0    472120   0% /sys/fs/cgroup
/dev/mmcblk0p6     67434   24154     43280  36% /boot
tmpfs              94424       0     94424   0% /run/user/1000

 

2017-11-29_RPi3 StarTech mSATA Adapter Card + Samsung 850 EVO mSATA 250GB SSD Assembled_IMG_5439 edit.jpeg

Edited by f11
Link to comment
Share on other sites

Downloaded the .zip file from post, copied your crt.py over Matt's, restarting weewx and waiting ...  and messages, updates, and all the rest !!

:-) :-) :-) :-) :-)

Here's the WP log (hmm, I got these lines twice ... anyway) :

2020-01-24 13:15:22,190 [Thread-1  ] [DEBUG] Got some WeeWX data
2020-01-24 13:15:22,192 [Thread-1  ] [INFO ]  - Set temperature driver ST to 4.3
2020-01-24 13:15:22,193 [Thread-1  ] [INFO ] Updating Driver temperature - ST: 4.3, uom: 4
2020-01-24 13:15:22,195 [Thread-1  ] [INFO ]  - Set temperature driver GV0 to -0.9
2020-01-24 13:15:22,196 [Thread-1  ] [INFO ] Updating Driver temperature - GV0: -0.9, uom: 4
2020-01-24 13:15:22,198 [Thread-1  ] [INFO ]  - Set humidity driver ST to 69
2020-01-24 13:15:22,200 [Thread-1  ] [INFO ] Updating Driver humidity - ST: 69.0, uom: 51
2020-01-24 13:15:22,202 [Thread-1  ] [INFO ]  - Set temperature driver GV1 to 4.3
2020-01-24 13:15:22,202 [Thread-1  ] [INFO ] Updating Driver temperature - GV1: 4.3, uom: 4
2020-01-24 13:15:22,204 [Thread-1  ] [INFO ]  - Set light driver GV0 to 109
2020-01-24 13:15:22,205 [Thread-1  ] [INFO ] Updating Driver light - GV0: 109.0, uom: 74
2020-01-24 13:15:22,207 [Thread-1  ] [INFO ]  - Set temperature driver GV4 to 20.8
2020-01-24 13:15:22,207 [Thread-1  ] [INFO ] Updating Driver temperature - GV4: 20.8, uom: 4
2020-01-24 13:15:22,209 [Thread-1  ] [INFO ]  - Set humidity driver GV0 to 34
2020-01-24 13:15:22,210 [Thread-1  ] [INFO ] Updating Driver humidity - GV0: 34.0, uom: 51
2020-01-24 13:15:22,212 [Thread-1  ] [INFO ]  - Set pressure driver GV1 to -2.0
2020-01-24 13:15:22,213 [Thread-1  ] [INFO ] Updating Driver pressure - GV1: -2.0, uom: 25
2020-01-24 13:15:22,215 [Thread-1  ] [INFO ]  - Set pressure driver ST to 1003.2
2020-01-24 13:15:22,216 [Thread-1  ] [INFO ] Updating Driver pressure - ST: 1003.2, uom: 117
2020-01-24 13:15:22,218 [Thread-1  ] [INFO ]  - Set wind driver GV4 to 0.4
2020-01-24 13:15:22,220 [Thread-1  ] [INFO ] Updating Driver wind - GV4: 0.4, uom: 32
2020-01-24 13:15:22,222 [Thread-1  ] [INFO ]  - Set wind driver GV1 to 0.4
2020-01-24 13:15:22,224 [Thread-1  ] [INFO ] Updating Driver wind - GV1: 0.4, uom: 32
2020-01-24 13:15:22,226 [Thread-1  ] [INFO ]  - Set wind driver GV0 to SE
2020-01-24 13:15:22,227 [Thread-1  ] [DEBUG]   - setDriver failed 11  -> wind could not convert string to float: 'SE'
2020-01-24 13:15:22,229 [Thread-1  ] [INFO ] "POST /weewx HTTP/1.1" 200 -

I may have the failed 'wind' issue because I grabbed the wrong value - I see I grabbed the ordinal wind dir, not the bearing.

Edited by f11
Link to comment
Share on other sites

Yup, consistently getting two sets of identical log file entries for each update, varying only by time stamp in the msec field.

Also, wind direction, both ordinal and bearing, are having problems converting ordinals (SE, WSW, - - - , etc) and bearings=NULL / - - - into floats, understandably.

2020-01-24 14:20:21,995 [Thread-1  ] [INFO ]  - Set wind driver GV0 to NULL
2020-01-24 14:20:21,995 [Thread-1  ] [DEBUG]   - setDriver failed 7  -> wind could not convert string to float: 'NULL'

 

Link to comment
Share on other sites

4 minutes ago, f11 said:

Yup, consistently getting two sets of identical log file entries for each update, varying only by time stamp in the msec field.

Also, wind direction, both ordinal and bearing, are having problems converting ordinals (SE, WSW, - - - , etc) and bearings=NULL / - - - into floats, understandably.


2020-01-24 14:20:21,995 [Thread-1  ] [INFO ]  - Set wind driver GV0 to NULL
2020-01-24 14:20:21,995 [Thread-1  ] [DEBUG]   - setDriver failed 7  -> wind could not convert string to float: 'NULL'

 

I'm not sure I understand what you mean by two sets of identical log entries.  Do mean the whole block between "Got some WeeWx data"  and "POST /weewx HTTP/1.1 200" is duplicated?  That would imply that you are actually getting two requests from WeeWx (or have two copies running maybe).   Or are you referring to the 

1 hour ago, f11 said:

 


2020-01-24 13:15:22,192 [Thread-1  ] [INFO ]  - Set temperature driver ST to 4.3
2020-01-24 13:15:22,193 [Thread-1  ] [INFO ] Updating Driver temperature - ST: 4.3, uom: 4

 

two lines above?  Those are normal. The first is my code saying it's telling Polyglot to send an update to the ISY and the second is Polyglot saying it is sending the up to the ISY.  I tend to sometimes over log things.  I've started converting my node servers over to have user configurable logging along with some other improvements but I've not yet cleaned up this node server.

Link to comment
Share on other sites

Yes, 'the whole block between "Got some WeeWx data"  and "POST /weewx HTTP/1.1 200" is duplicated'.  The two blocks are identical in all regards, except for their timestamps, which only differ in msec.

Checked for multiple instances of weewx using 'ps aux' and 'htop' -> nope, just one that's visible ... checking for multiple crt extensions in weewx using wee_extensions --list -> nope, just one is listed.

Did an update and dist-upgrade of the weewx RPi just in case - it goes long periods without restarting or being updated) - then rebooted Polisy, and rebooted ISY so everything is starting out in a somewhat known state.  Everything works like a damn - except for the two blocks of identical log entries.  Weird.

Is there a way to temporarily disable weewx-crt without stopping-starting everything?  If I have some kind of double-instance action going on, maybe I can shut down the one I can see to help me find the one I can't.

Since you're not seeing this double-log effect, its obviously at my end.  I'll keep looking.

But otherwise, everything looks great with WeatherPoly and your mod'd version of weewx-crt : updates come in with only slight latency from the update of realtime.txt to arrival in WP's log, and the data transfers across to ISY smoothly and appears on the ISY screen as expected. 

The only issue I've seen so far is the failed conversion of realtime.txt 'none/NULL/- - -/N NE E SW etc' strings to values in WP, clearly an artifact of dealing with the incoming file fields instead of live data.  Although I've only seen it show up in wind direction (both ordinal and bearing) custom parameter fields, its likely to be common to all converted fields.  I was going to test that next, just assigning random fields to various custom parameters to see if WP can handle unexpected inputs gracefully.

Link to comment
Share on other sites

I'm not seeing the duplicates. I do get updates every few seconds but the messages from WeeWX in the /var/log/messages correspond to the log entries in the WeatherPoly log.

Currently, WeatherPoly handles bad data by reporting the error and ignoring it. Anything that's not numeric is considered bad data.  

Link to comment
Share on other sites

Ah - you've set your system to update on 'loop' (~3 sec period) while mine is set to update on 'archive' (300 sec period).

Ok, so the WP log message regarding wind direction isn't really an 'error', its just flagging what it considers bad data.  In that case, I've run across zero issues so far.  weewx-crt starts and stops gracefully in tandem with weewx (which I would expect), and other than logging activity just hums along in the background.  So far, things are working extremely well.

Link to comment
Share on other sites

Think I found a culprit in weewx.conf:

[Engine]
    [[Services]]
        . . . 
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate, user.crt.CumulusRealTime
        archive_services = weewx.engine.StdArchive, user.crt.CumulusRealTime
        . . . 

'user.crt.CumulusRealTime' is listed at the end of both 'process_services' and 'archive_services'.  Obviously this is my doing, but not sure if I did it for a reason, or just fat-fingered the entry.  Looking back in my archives, this doubling of service entries first shows up in my 24Jan 11:36AM update of weewx.conf.  By that time, I was working with a lot of files and folders, so I'm guessing it was a fat-finger snafu.  Doesn't matter.

I fixed the [Engine] entries, stopped and re-started weewx, and watched the logs -> ??SUCCESS ??!!  The next update showed up in the WP log (with what seemed to be a lot less latency) as only one block of entries.

Glad that's sorted!  THANKS so much for your help with this !!  I'd have NEVER sorted all this out on my own.  I also appreciate the opportunity to tag along with a professional coder, I learned a lot.  I'll keep poking at WP and weewx-crt and let you know what I find.

Best Regards, Rod

Link to comment
Share on other sites

Bob, just an update ...

I've been poking at WeatherPoly and your weewx-crt mod'd code, trying to break things, for the past several days.  Its really robust, and so far seems impervious to unexpected data or mis-configurations.  Its been running without interruption now since last Friday, Jan 24th, and hasn't interfered with weewx in any way that I've seen.

I'm presently transferring 3 wind, 2 pressure, 2 humidity, 1 light (radiation sensor), and 4 temperature values from my Davis Vantage Pro 2 station to WeatherPoly via the weewx-crt mod'd code - all have been updating 24/7 since Friday without issue.  Same report for WeatherPoly regarding its modifications to handle the auto-sent weewx-crt file.  Everything just works, even if I reboot the RPi running weewx, bounce the power on Polisy, or interrupt the data coming from the weather station.

The only comment I'd add is that I look forward to being able to reduce WP log entries via configurable WP setting, as you said you were considering.  Now that everything is running smoothly, the number of logged entries can be cut back considerably, in both the Polisy and WP logs.

If there's any specific kind of testing you want me to do on my system, let me know.  Otherwise, I'm a pretty happy camper !  Thank you !

Rod

Link to comment
Share on other sites

  • 3 weeks later...

I'm very glad I found this post about your efforts with WeeWx because I was just about to go down that rabit hole with my system. I have WeeWx running with a  DIY weather station and I'm currently uploading to PWSweather and Wunderground. However, with the  climate module being discontinued, I  am looking for alternatives. I was going to have WeeWx output JSON via the method I found here: https://hoke.org/john/2018/05/using-weewx-for-home-automation/ and try to write a node server to parse it, but it seems like you gentlemen found a solution that works. I will see if I can follow your progress posts and replicate what you have done on my system.

Thanks!

Steve

  • Like 1
Link to comment
Share on other sites

And to update the status of the modified weewx-crt extension.  Matt rejected the pull request to add this ability to his extension and he had some pretty good reasons. Mainly he thought it was a bit outside the scope of his extension and would rather see it as a separate extension.   He also noted that as it's currently written, a problem with the HTTP request could block WeeWx for an extended amount of time which is not a good thing. 

I understand his concerns and agree that the it should be fixed so that it can't block WeeWX.  However, I'm not really using WeeWX and don't really want to try and maintain an extension for it.  If someone who is willing to maintain an extension wants to step up, I'd be happy help modify my version of the weewx-crt so that it can be a standalone extension.   

In the meantime, since this is working, I'll keep my fork up there on github for anyone that needs it.

Link to comment
Share on other sites

Not sure if I missed instructions somewhere, but finally figured out I had to map the nodes form this list:  https://github.com/bpaauwe/WeatherPoly to positions in the url.  I can't see the url coming in so I looked it up in CRT.py.  

I added your changes to my CRT.py file that I use to publish data for a local weather page using the realtime.txt.  

I can create the variables for yesterday records you have commented out if anyone would like them.  I'm mostly writing this to document what I had to do to get it working.  If mapping temperature-main = 2 was incorrect then I'd like to know the right method.  Also, in the CRT.py temp is line 3 but the first 2 or dates so trail and error got me there.

I have a request.  I actually pull in some other weather data through some extensions to weewx. In particular I pull in the lake level from the geological site.

Could we add node category and maybe call it environment.  I'd like variables for lakelevel (current, max_year, min_year, max_month, min_month, min_day, max_day) . It could just be water level so it is more generic.  All these are in feet of elevation so convert from meter to feet ...

I could see other variables like snow in this category.  If I still had polyglot on my raspberry instead of the polisy I'd hack the files to see I could get it working. 

Link to comment
Share on other sites

18 hours ago, bigDvette said:

Not sure if I missed instructions somewhere, but finally figured out I had to map the nodes form this list:  https://github.com/bpaauwe/WeatherPoly to positions in the url.  I can't see the url coming in so I looked it up in CRT.py.  

I added your changes to my CRT.py file that I use to publish data for a local weather page using the realtime.txt.  

I can create the variables for yesterday records you have commented out if anyone would like them.  I'm mostly writing this to document what I had to do to get it working.  If mapping temperature-main = 2 was incorrect then I'd like to know the right method.  Also, in the CRT.py temp is line 3 but the first 2 or dates so trail and error got me there.

I have a request.  I actually pull in some other weather data through some extensions to weewx. In particular I pull in the lake level from the geological site.

Could we add node category and maybe call it environment.  I'd like variables for lakelevel (current, max_year, min_year, max_month, min_month, min_day, max_day) . It could just be water level so it is more generic.  All these are in feet of elevation so convert from meter to feet ...

I could see other variables like snow in this category.  If I still had polyglot on my raspberry instead of the polisy I'd hack the files to see I could get it working. 

The indexing in WeatherPoly is zero based while the list in crt.py starts at one.  So the index is 1 less than the line #.   That probably isn't documented anywhere.

While it's possible to add more node types and more data values, it does take a bit of effort and because the way Polyglot and the ISY are currently architected, everything tends to be fairly static.  What that means is that adding in something like lake level is probably not something that would be used by very many people (I.E. probably just you at this time). If there was a way to do additions like this with just changes to the configuration, I'd do that.  But I currently don't have the time to come up with a way to add user defined nodes/values.

BTW, it's just about as easy to hack on the files in a Polisy as it is in a RPi. 

 

Link to comment
Share on other sites

wow, this thing does a lot of info logging that I can't turn off. My polisy becomes unresponsive when running WeatherPoly.  My guess is that my crt.py is bound to LOOP and not archive.

I ended up copying CRT.py and modifying the class and section so I could have 2 instances running (1 bound to loop for my realtime weather station and 1 bound to archive for ISY).  

not sure if slowness was due to the logging or the actual processing of variables.

Link to comment
Share on other sites

I didn't think it was doing that much logging. Some of my other node servers do a lot more.  I do have the goal to add configurable logging to all of them eventually so I added it to WeatherPoly and pushed version 0.1.9 

I would think that it would take a lot of processing to make the Polisy become unresponsive. How often does loop execute?  I could see if it's once a second and you're sending a bunch of values.  Looks like I do have it always send an update even if the value hasn't changed so that's something I could optimize.

Link to comment
Share on other sites

I think not sending values unless they change is a good thing in general. But values should be sent every so often even if they do not change in case ISY is rebooted so the values will get updated after some amount of time. Maybe every 5 or 10 minutes or a configurable parameter.

 

While I am making suggestions I don't think temperatures should be NNN.NN. I think they should be NNN.N.

Who needs to know if the temperature changed by 100th of a degree?

This can cause programs to trigger every time the value is updated because it changes between say 81.11 and 81.12.

 

Edited by jca001
Link to comment
Share on other sites

Davis weather stations have. 2 second loop packet so it was sending very variable 12, every second.  I commented out all the logger.info and it still streamed a lot.  I think it is a polyglot setting potentially as harmony also has a tendency to be chatty.

i had asked the question if you wanted help with the custom crt.py in terms of getting other variables.  I store my lake level in weewx and pull it out and do math on the variables so I’ve figured out the sql methods.

i noticed you I had some comments in the file to come back later.

Link to comment
Share on other sites

15 hours ago, jca001 said:

I think not sending values unless they change is a good thing in general. But values should be sent every so often even if they do not change in case ISY is rebooted so the values will get updated after some amount of time. Maybe every 5 or 10 minutes or a configurable parameter.

 

While I am making suggestions I don't think temperatures should be NNN.NN. I think they should be NNN.N.

Who needs to know if the temperature changed by 100th of a degree?

This can cause programs to trigger every time the value is updated because it changes between say 81.11 and 81.12.

 

Are you commenting about the same node server as above, WeatherPoly?  I just looked at it and all temperature values are rounded to 1 decimal place and the node definition specifies temperature with a precision of 1 decimal place. It should not be reporting temperatures to the 100th of a degree.

Link to comment
Share on other sites

14 hours ago, bigDvette said:

Davis weather stations have. 2 second loop packet so it was sending very variable 12, every second.  I commented out all the logger.info and it still streamed a lot.  I think it is a polyglot setting potentially as harmony also has a tendency to be chatty.

i had asked the question if you wanted help with the custom crt.py in terms of getting other variables.  I store my lake level in weewx and pull it out and do math on the variables so I’ve figured out the sql methods.

i noticed you I had some comments in the file to come back later.

Doing something every second or two could be the issue.  I did just push a new version with configurable logging so that may help some, but it's probably the http rest calls to the ISY that are really the issue.  There isn't currently a way to batch them up so every node value update is a separate http request and doing many every second is probably not good for either the Polisy or the ISY.

So I'm not really maintaining a custom crt.py as I don't use WeeWX and don't want to try an maintain an extension for something I don't use.  The author of the crt.py extension did look at the changes and had a few issues with them.  1) he didn't think they really fit with the goals he had for the extension and suggested it be a separate extension. 2) it shouldn't be doing the http requests from inside the loop. His suggestion was to spawn a thread to do the http request, that way it wouldn't block other loop processing if it takes a while or times out.  Moving it archive instead of loop is probably a good idea too. 

If you're willing to maintain a WeeWX extension for this, I am willing to make any changes necessary to WeatherPoly to continue to work with it.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.

  • Recently Browsing

    • No registered users viewing this page.
  • Forum Statistics

    • Total Topics
      36.5k
    • Total Posts
      367.7k
×
×
  • Create New...