jca001 Posted February 26, 2020 Share Posted February 26, 2020 4 minutes ago, bpwwer said: Are you commenting about the same node server as above, WeatherPoly? I just looked at it and all temperature values are rounded to 1 decimal place and the node definition specifies temperature with a precision of 1 decimal place. It should not be reporting temperatures to the 100th of a degree. No. I was just making a general statement about any thing that has a temperature values, weather or thermostats. Some others that I think should have less precision, no fraction are MPH and humidity. Link to comment Share on other sites More sharing options...
jca001 Posted February 27, 2020 Share Posted February 27, 2020 23 hours ago, bpwwer said: Are you commenting about the same node server as above, WeatherPoly? I just looked at it and all temperature values are rounded to 1 decimal place and the node definition specifies temperature with a precision of 1 decimal place. It should not be reporting temperatures to the 100th of a degree. I know I have already replied to this, but I was looking at my RPI Node Server and noticed for the last 2 days OpenWeatherMap has showed offline. I just click on start and it is OK. I don't see an error, it just seems to stop. I can provide the log file if you want to look at it. Looking at the values on the AC it has temperatures and wind speed with NNN.NN, which is what I was talking about in general. I think they should be whole numbers. I cannot really see somebody want to trigger a program if the temperature was say above 88.6 F. Maybe 1-digit fraction precision is needed for "C". Link to comment Share on other sites More sharing options...
bpwwer Posted February 27, 2020 Author Share Posted February 27, 2020 1 hour ago, jca001 said: I know I have already replied to this, but I was looking at my RPI Node Server and noticed for the last 2 days OpenWeatherMap has showed offline. I just click on start and it is OK. I don't see an error, it just seems to stop. I can provide the log file if you want to look at it. Looking at the values on the AC it has temperatures and wind speed with NNN.NN, which is what I was talking about in general. I think they should be whole numbers. I cannot really see somebody want to trigger a program if the temperature was say above 88.6 F. Maybe 1-digit fraction precision is needed for "C". Typically, it stops (or crashes) when it gets bad data from the service. If the log shows what caused it, I'll update the node server to trap that bad data and not crash, but the initial assumption was that the service would provide good data. I don't disagree with you about the precision of some of the values. For services like OpenWeatherMap, I'm just forwarding on the data, as the service provides it, to the ISY. If I round/truncate the data I could then get the opposite complaint, why doesn't the data reported by the ISY match what the service is reporting? My thought is that since the node server is basically just a way of transferring the data from a service/device to the ISY it probably shouldn't be corrupting that data unless there's a really good reason. Link to comment Share on other sites More sharing options...
jca001 Posted February 27, 2020 Share Posted February 27, 2020 6 minutes ago, bpwwer said: Typically, it stops (or crashes) when it gets bad data from the service. If the log shows what caused it, I'll update the node server to trap that bad data and not crash, but the initial assumption was that the service would provide good data. I don't disagree with you about the precision of some of the values. For services like OpenWeatherMap, I'm just forwarding on the data, as the service provides it, to the ISY. If I round/truncate the data I could then get the opposite complaint, why doesn't the data reported by the ISY match what the service is reporting? My thought is that since the node server is basically just a way of transferring the data from a service/device to the ISY it probably shouldn't be corrupting that data unless there's a really good reason. It has stopped at different time. The day before it was around 06:?? and then at about 20:30. Nothing specific happened on my network that I am aware of and my cable modem, router, switch and RPI are all on an UPS. I know it would take more coding, but maybe it would be good if there was an option to change the precision to what the user wanted to see. I just think it is bad to cause the ISY to trigger a program each time a value changes by 100th or even 10th when the test may be for a whole value. Maybe I am wrong about this but I am pretty sure I had a test program that just incremented a counter when the temperature was above say 80 F and it counted a lot probably each time it changed to 80.1, 80.2, 80.3 and then back to 80.2, etc. This is just my opinion, but other may have different ones. Link to comment Share on other sites More sharing options...
bpwwer Posted February 27, 2020 Author Share Posted February 27, 2020 5 minutes ago, jca001 said: It has stopped at different time. The day before it was around 06:?? and then at about 20:30. Nothing specific happened on my network that I am aware of and my cable modem, router, switch and RPI are all on an UPS. I know it would take more coding, but maybe it would be good if there was an option to change the precision to what the user wanted to see. I just think it is bad to cause the ISY to trigger a program each time a value changes by 100th or even 10th when the test may be for a whole value. Maybe I am wrong about this but I am pretty sure I had a test program that just incremented a counter when the temperature was above say 80 F and it counted a lot probably each time it changed to 80.1, 80.2, 80.3 and then back to 80.2, etc. This is just my opinion, but other may have different ones. If the node server is stopping there should be a trace dump in the node server's log when that happens. If you send me that, I can update the node server to trap and continue when whatever caused it happens again. It's likely nothing on your side that's causing it. Most likely is bad or missing data from OpenWeatherMap. 1 Link to comment Share on other sites More sharing options...
jca001 Posted February 27, 2020 Share Posted February 27, 2020 The next time it shows disconnected I will send you the log file. Link to comment Share on other sites More sharing options...
f11 Posted March 2, 2020 Share Posted March 2, 2020 (edited) Bob. trying to install your Polisy WeatherPoly "UPDATE" from 0.1.8 to 0.1.9, but keep getting "pullRepo: undefined" instead of report of successful update. Is this something I'm doing incorrectly on my side? The Polisy "pkg info" command inside ssh, and my ISY version, show: e2fsprogs-libuuid-1.45.5 UUID library from e2fsprogs package git-2.25.1 Distributed source code management tool pkg-1.13.2 Package manager polyglot-2.2.9_5 Polyglot v2 for UDI ISY 5.x.x udx-1.0.26_27 Polisy Admin Service ISY Version: 5.0.16B When I click on "UPDATE" in NodeServer Store, the pop-up shows "Yes", "No", and "Cancel". Your ReadME instructions on Upgrading say to choose "Cancel" for Polyglot 2.0.35, and "No" for Polyglot 2.0.36. When restarting WeatherPoly, I see 2020-03-01 22:04:58,189 [MainThread] [INFO ] UDI Polyglot v2 Interface 2.0.34 Starting... in your log, so I guess I have Polyglot 2.0.34. I've tried Dashboard System -> Check for Polisy Updates which reports I'm up to date. Also tried 'sudo pkg update && sudo pkg upgrade' from the command line, and it also reported "Your packages are up to date". Hmm I didn't check these forums for notes on Polyglot updates, so I'll do that next. [ edit: wow, I must be tired. Obviously Polyglot 2.2.9_5 is a LOT more recent than 2.0.36 ... so ignore all that blah-blah about versions. But my basic question remains: when the Update dialog pops up with the "Yes" "No" Cancel" choices, which do I select to update WeatherPoly to 0.1.9 (I'd love to cut back a bit on the INFO logging)? If I read your note correctly, I should choose "No"? ] On a side note, I'm not sure I have the coding wherewithal to take on the new CRT.py weewx extension you referred to after a conversation with Matt - but I'm willing to look into it. I certainly agree with Matt's assessment - anything that threatens weewx's legendary robustness is something to be avoided, although after over a month of 7/24 running Matt's extension and your weatherpoly modifications on my system there's been zero glitches. So it looks like the suggestion was to put the http server code in its own thread to prevent blocking weewx. Let me know what your back-of-the-envelope take is on such an extension to weewx and I'll take a crack at it. Rod Edited March 2, 2020 by f11 Link to comment Share on other sites More sharing options...
DennisC Posted March 2, 2020 Share Posted March 2, 2020 Yes, there is some bug UD is working on that is causing this. The solution for now is to uninstall the Node Server and then reinstall the updated version. It might be a good idea to reboot in-between. When installing the update, be sure to install it to the same spot and none of the programs will be impacted. Link to comment Share on other sites More sharing options...
bpwwer Posted March 2, 2020 Author Share Posted March 2, 2020 10 hours ago, f11 said: [ edit: wow, I must be tired. Obviously Polyglot 2.2.9_5 is a LOT more recent than 2.0.36 ... so ignore all that blah-blah about versions. But my basic question remains: when the Update dialog pops up with the "Yes" "No" Cancel" choices, which do I select to update WeatherPoly to 0.1.9 (I'd love to cut back a bit on the INFO logging)? If I read your note correctly, I should choose "No"? ] On a side note, I'm not sure I have the coding wherewithal to take on the new CRT.py weewx extension you referred to after a conversation with Matt - but I'm willing to look into it. I certainly agree with Matt's assessment - anything that threatens weewx's legendary robustness is something to be avoided, although after over a month of 7/24 running Matt's extension and your weatherpoly modifications on my system there's been zero glitches. So it looks like the suggestion was to put the http server code in its own thread to prevent blocking weewx. Let me know what your back-of-the-envelope take is on such an extension to weewx and I'll take a crack at it. Rod That part of the README was taken from the sample node server. I believe the newer versions of Polyglot automatically handle the profile updates so yes or no doesn't really matter anymore. I think that changing the extension to use a thread would be pretty simple. I believe just changing the call to the post_data() function directly in handle_data(), it would spawn a thread that runs the post_data() function. Link to comment Share on other sites More sharing options...
jca001 Posted March 2, 2020 Share Posted March 2, 2020 On 2/27/2020 at 1:22 PM, bpwwer said: If the node server is stopping there should be a trace dump in the node server's log when that happens. If you send me that, I can update the node server to trap and continue when whatever caused it happens again. It's likely nothing on your side that's causing it. Most likely is bad or missing data from OpenWeatherMap. I had not checked the values of OpenWeatherMap or looked at the NS for a couple of days. It has stopped again but there is no trace back. Here are the last lines in the log file. OpenWeatherMap_log.txt Link to comment Share on other sites More sharing options...
bpwwer Posted March 2, 2020 Author Share Posted March 2, 2020 3 hours ago, jca001 said: I had not checked the values of OpenWeatherMap or looked at the NS for a couple of days. It has stopped again but there is no trace back. Here are the last lines in the log file. OpenWeatherMap_log.txt 4.57 kB · 2 downloads That looks like it never returned from the query to get the forecast data. I'm curious if the node server is actually running when this happens. You should be able to check for the running process on the system with a 'ps auxww | grep owm' command line. If there's a python3 owm.py process running then the node sever is still running, just hung. If it's not there then it looks like something killed the node server. Link to comment Share on other sites More sharing options...
f11 Posted March 2, 2020 Share Posted March 2, 2020 4 hours ago, bpwwer said: I think that changing the extension to use a thread would be pretty simple. I believe just changing the call to the post_data() function directly in handle_data(), it would spawn a thread that runs the post_data() function. I've worked with python threads on some other at-home projects, so I'm familiar with configuring and launching them, but not overly familiar with the 'gotcha's' they may introduce (locking. etc). As you say, tho, it appears the heavy lifting has already been done by your code and that of Matt, so it shouldn't be much of a stretch. (Hmm, guess I better get up to speed on GitHub if I'm going to tackle something formal.) Fortunately, I can test a prototype on my own system before offering it to an unsuspecting public. Back atcha later on this. Oh, and thx to you and DennisC for the update on updating. Link to comment Share on other sites More sharing options...
jca001 Posted March 3, 2020 Share Posted March 3, 2020 3 hours ago, bpwwer said: That looks like it never returned from the query to get the forecast data. I'm curious if the node server is actually running when this happens. You should be able to check for the running process on the system with a 'ps auxww | grep owm' command line. If there's a python3 owm.py process running then the node sever is still running, just hung. If it's not there then it looks like something killed the node server. I will do that if it happens again. I have already clicked on START and it is running for now. I my other topic I stated I now have ISY Programs that will send me an email when the status indicated disconnected so I will know when it stops sooner. Link to comment Share on other sites More sharing options...
DennisC Posted March 19, 2020 Share Posted March 19, 2020 (edited) @bpwwer Hi Bob, I updated today to the latest WheatherFlowPoly and had to issues. In the log, I am seeing the following : 2020-03-19 15:05:20,056 [NodeServer] [INFO ] Starting WeatherFlow Node Server 2020-03-19 15:05:20,057 [Interface ] [DEBUG] -- configuration is valid 2020-03-19 15:05:20,058 [NodeServer] [ERROR] Found saved log level 30 2020-03-19 15:05:20,059 [NodeServer] [ERROR] saving log level to customdata 2020-03-19 15:05:20,060 [NodeServer] [INFO ] Sending customData to Polyglot. 2020-03-19 15:05:20,198 [NodeServer] [INFO ] set_logging_level: Setting log level to 30 2020-03-19 15:06:09,341 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:07:09,206 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:08:09,268 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:09:09,335 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:10:09,184 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:11:09,253 [Thread-1 ] [ERROR] Failure in processing AIR data In the AC, I lost all data for Precipitation. All other fields have data, but appear to be frozen and are not updating. This is supported by no new data in the Polyglot log. Any suggestions? Update: Apparently, the log is not adding data when it is open. After closing and opening the log, I see additional Failure in processing AIR data. dc Thanks Dennis Edited March 19, 2020 by DennisC Added info. Link to comment Share on other sites More sharing options...
bpwwer Posted March 19, 2020 Author Share Posted March 19, 2020 1 hour ago, DennisC said: @bpwwer Hi Bob, I updated today to the latest WheatherFlowPoly and had to issues. In the log, I am seeing the following : 2020-03-19 15:05:20,056 [NodeServer] [INFO ] Starting WeatherFlow Node Server 2020-03-19 15:05:20,057 [Interface ] [DEBUG] -- configuration is valid 2020-03-19 15:05:20,058 [NodeServer] [ERROR] Found saved log level 30 2020-03-19 15:05:20,059 [NodeServer] [ERROR] saving log level to customdata 2020-03-19 15:05:20,060 [NodeServer] [INFO ] Sending customData to Polyglot. 2020-03-19 15:05:20,198 [NodeServer] [INFO ] set_logging_level: Setting log level to 30 2020-03-19 15:06:09,341 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:07:09,206 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:08:09,268 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:09:09,335 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:10:09,184 [Thread-1 ] [ERROR] Failure in processing AIR data 2020-03-19 15:11:09,253 [Thread-1 ] [ERROR] Failure in processing AIR data In the AC, I lost all data for Precipitation. All other fields have data, but appear to be frozen and are not updating. This is supported by no new data in the Polyglot log. Any suggestions? Update: Apparently, the log is not adding data when it is open. After closing and opening the log, I see additional Failure in processing AIR data. dc Thanks Dennis It doesn't like something about the data the Air is sending. If the log stopped updating at that point, that would indicate that it's not seeing any data from the hub. What about the hub seconds since seen value on the main controller node? Is that increasing at all? It should increase by 60 every minute. If it's not updating then the node server is not running or at least not running in a way that is still connected to Polyglot. Is the Sky data updating? Make sure you set the log level to info or debug or you'll just get the error message and not any of the other log message. Link to comment Share on other sites More sharing options...
DennisC Posted March 19, 2020 Share Posted March 19, 2020 36 minutes ago, bpwwer said: It doesn't like something about the data the Air is sending. If the log stopped updating at that point, that would indicate that it's not seeing any data from the hub. What about the hub seconds since seen value on the main controller node? Is that increasing at all? It should increase by 60 every minute. If it's not updating then the node server is not running or at least not running in a way that is still connected to Polyglot. Is the Sky data updating? Make sure you set the log level to info or debug or you'll just get the error message and not any of the other log message. I cycled power for the hub just to be sure. I also set log level to debug. I'm seeing Sky data in the AC and the log. I am still getting the error for the Air data. I trippled check the Air configuration information in the Node Server and it is correct, including the s/n. In the log I am seeing the heartbeat, but hub seconds since update is not increasing. Not sure if it will help, but I will PM you a log package. dc Link to comment Share on other sites More sharing options...
bpwwer Posted March 19, 2020 Author Share Posted March 19, 2020 I just pushed a new version with additional error messages and debug. I'm not seeing any issues like this on my system, but something in either air data or in my processing of it is failing and hopefully the additional messages will provide the details. Link to comment Share on other sites More sharing options...
bpwwer Posted March 19, 2020 Author Share Posted March 19, 2020 I just added a new weather service node server for AERISWeather.com. AERISWeather is the service that currently provides the data source for the climate module. However, they do not have a free plan available. For folks that have a personal weather station that push data to PWSWeather.com, you can request a free AERISWeather account through their partner program. This allows you to access the data you upload to PWSWeather.com and pull down AERIS forecast data. This should also work with any of the paid AERIS subscription plans also. 2 Link to comment Share on other sites More sharing options...
glarsen Posted March 19, 2020 Share Posted March 19, 2020 Thanks @bpwwer. There is also an Aeris parser for the Rainmachine irrigation controller at https://github.com/ve7gel/rainmachine-developer-resources/blob/master/sdk-parsers/RMParserFramework/parsers/aeris-parser.py that is based on the same contributor option. Link to comment Share on other sites More sharing options...
DennisC Posted March 19, 2020 Share Posted March 19, 2020 27 minutes ago, bpwwer said: I just pushed a new version with additional error messages and debug. I'm not seeing any issues like this on my system, but something in either air data or in my processing of it is failing and hopefully the additional messages will provide the details. New version hasn't hit the store yet. I will try again a little later. Link to comment Share on other sites More sharing options...
bpwwer Posted March 19, 2020 Author Share Posted March 19, 2020 I think the store updates once an hour, looks like it's currently scanning at 43 minutes after the hour. Should be there now. Link to comment Share on other sites More sharing options...
DennisC Posted March 19, 2020 Share Posted March 19, 2020 29 minutes ago, bpwwer said: I think the store updates once an hour, looks like it's currently scanning at 43 minutes after the hour. Should be there now. I did the update. Right now, I am not seeing the error. I will let you know what happens. Is there anyway of re-entering the missing rain data? Link to comment Share on other sites More sharing options...
bpwwer Posted March 19, 2020 Author Share Posted March 19, 2020 21 minutes ago, DennisC said: I did the update. Right now, I am not seeing the error. I will let you know what happens. Is there anyway of re-entering the missing rain data? It stores the rain data in the node server's customData which is stored in the polyglot mongodb. So, in theory, you could stop the node server, update that data in the database, and then restart the node server and it should be visible. I'm not sure why it failed to save (or maybe reset) the data. It might be time for me to think about adding the ability to pull that data from the WeatherFlow server when the node server starts up. Link to comment Share on other sites More sharing options...
DennisC Posted March 19, 2020 Share Posted March 19, 2020 5 minutes ago, bpwwer said: It stores the rain data in the node server's customData which is stored in the polyglot mongodb. So, in theory, you could stop the node server, update that data in the database, and then restart the node server and it should be visible. I'm not sure why it failed to save (or maybe reset) the data. It might be time for me to think about adding the ability to pull that data from the WeatherFlow server when the node server starts up. I'm thinking that could be very helpful. I would also like to have a way to store rain check rain values. Link to comment Share on other sites More sharing options...
glarsen Posted March 19, 2020 Share Posted March 19, 2020 (edited) 1 hour ago, bpwwer said: I just added a new weather service node server for AERISWeather.com. AERISWeather is the service that currently provides the data source for the climate module. However, they do not have a free plan available. For folks that have a personal weather station that push data to PWSWeather.com, you can request a free AERISWeather account through their partner program. This allows you to access the data you upload to PWSWeather.com and pull down AERIS forecast data. This should also work with any of the paid AERIS subscription plans also. @bpwwer, I installed this NS on my Polisy and other than having to manually group the forecast periods in the AC, it all looks good using the data from my PWS. I am curious though about the et0 numbers in the forecasts. Are they acquired from Aeris, or are they calculated? I'm interested to compare them to the numbers from both my Davis VP2+ and our local Govt Agricultural office (which sometimes align, and often don't). Also, what is the "Climate Coverage" driver ? It's blank in all forecasts and local data. Thanks! Edited March 19, 2020 by glarsen Link to comment Share on other sites More sharing options...
Recommended Posts