Polyglot weather station node servers

[macjeff]

at first it would not work but deleted nodeserver and reinstalled and now working.

 

thanks

 

 

 

[macjeff]

New issue-   I know its most likely a data issue but once before I found a bug.  For that reason I noticed UNKNOWN under climate conditions  so wanted to send this to you.

 

[macjeff]

Hopefully this is not the wrong place to post about Weatherflow. 

I have your nodeserver hooked up to my TEMPEST.  I noticed that current precip and lightning are zero.  We have had an active day.   Tried restarting the nodeserver and nothing.  

Here are some screenshots from both the nodeserver and the tempest app.   Something must have changed in API?

 

 

[bpwwer]

WeatherFlow runs proprietary algorithms on the data sent by the station and reports that "corrected" data in the app. The node server does not do any correction of the data and simply reports what the stations has reported.

The correction is mainly done for rainfall and lightning.

However, I don't see anything "wrong" with the rain data.  Both are reporting a daily value of 0.6 inches. 

For lightning, there is a known issue in the Tempest firmware where it seems to miss a lot of lightning strikes and WeatherFlow pulls that data from other sources and reports it in the App.

If you configure the node server to pull the data from the WeatherFlow servers instead of from your Tempest directly, it will get the corrected values.  However, you are then depending on the cloud service.

[macjeff]

1 hour ago, bpwwer said:

WeatherFlow runs proprietary algorithms on the data sent by the station and reports that "corrected" data in the app. The node server does not do any correction of the data and simply reports what the stations has reported.

The correction is mainly done for rainfall and lightning.

However, I don't see anything "wrong" with the rain data.  Both are reporting a daily value of 0.6 inches. 

For lightning, there is a known issue in the Tempest firmware where it seems to miss a lot of lightning strikes and WeatherFlow pulls that data from other sources and reports it in the App.

If you configure the node server to pull the data from the WeatherFlow servers instead of from your Tempest directly, it will get the corrected values.  However, you are then depending on the cloud service.

I am on the TESTERS forum.  Is there a specific post I should chime in on?

I would rather get tempest data so let me know if there is a fix from them coming.

 

Jeff

[macjeff]

Purpleair nodeserver

 

can you add Query like you did to your others in the program commands

 

Set  Purpleairnodeserver to Query

 

thanks

[andrewm]

I just noticed an issue with the Climacell data being used in my irrigation system. The rainfall variable (GV6) is no longer being created in the forecast node.

The variable is still documented in the Poly's Readme file, but it looks like it was removed in the change to use the Climacell v4 API, though I didn't think I was using that; I certainly didn't have a v4 API key until just now, though I have now just upgraded the poly and created a v4 API key.

While creating my new API key I took a look at the v4 docs and do see that the v3 API was supposed to have been turned off on June 1st, which could explain why I'm just noticing issues now. Additionally, I don't see the forecast rainfall as a specific attribute, (though the API does have accumulated snow and icefall) which would seem a retrograde step if correct. But I'm not familiar with their API - maybe the precipitation accumulation is supposed to be calculated by aggregating the timeline values?

If this daily forecast datapoint isn't available in Climacell it leaves me needing to switch to a new weather provider. I do have a Tempest and did see references above to a Weatherflow forecast module. Is that something that's going to be available within a few months? In which case I'll wait for it, otherwise I'll just find something else.

Thanks.

[bpwwer]

6 hours ago, andrewm said:

I just noticed an issue with the Climacell data being used in my irrigation system. The rainfall variable (GV6) is no longer being created in the forecast node.

If this daily forecast datapoint isn't available in Climacell it leaves me needing to switch to a new weather provider. I do have a Tempest and did see references above to a Weatherflow forecast module. Is that something that's going to be available within a few months? In which case I'll wait for it, otherwise I'll just find something else.

Thanks.

Looks like they did remove it from the API.  I'll update the README, thanks for pointing that out.

The PG3 WeatherFlow node server has the forecast already done but I don't plan to upgrade the PG2 version.  Right now I'm only doing bug-fix changes to the PG2 based node servers as I try to get them all ported over to PG3.

[macjeff]

Bob,

I think this is a NOAA issue but I went to the nodeserver and data is not current.  So I restarted it and same thing.

I went to NOAA to look up my forecast and got this error

An error occurred while processing your request.

Reference #102.d1070f17.1626186736.1dfb5bbf

 

I tried entering cities like New York and it gives same error or no forecasts.

So I am guessing there is no data but logs look ok in the nodeserver to me.

By the time you read this they may have fixed it but can you give it a try?

[bpwwer]

1 hour ago, macjeff said:

Bob,

I think this is a NOAA issue but I went to the nodeserver and data is not current.  So I restarted it and same thing.

I went to NOAA to look up my forecast and got this error

An error occurred while processing your request.

Reference #102.d1070f17.1626186736.1dfb5bbf

 

I tried entering cities like New York and it gives same error or no forecasts.

So I am guessing there is no data but logs look ok in the nodeserver to me.

By the time you read this they may have fixed it but can you give it a try?

I started the node sever and it seemed to pull valid data so it could certainly have been a temporary outage.

Polyglot caches the more recent data sent to the ISY so if the node server stops receiving anything, that's what it will still get when queried.  By convention, the node server status reflects the status of the  node server itself, not the service or device it's connected to.  That's probably not right, but it's how most (all) of the polyglot based node servers work.  PG3 actually embeds this so that it automatically reports the node server executable running status by default.

[macjeff]

17 minutes ago, bpwwer said:

I started the node sever and it seemed to pull valid data so it could certainly have been a temporary outage.

Polyglot caches the more recent data sent to the ISY so if the node server stops receiving anything, that's what it will still get when queried.  By convention, the node server status reflects the status of the  node server itself, not the service or device it's connected to.  That's probably not right, but it's how most (all) of the polyglot based node servers work.  PG3 actually embeds this so that it automatically reports the node server executable running status by default.

Works now.  I wanted to report just in case but there was a temp outage.

With that said it was still cached on the old info and did not change (I had admin console open) so I did a query on it and it all updated fine.  But shouldnt it have updated auto?

 

 

[bpwwer]

26 minutes ago, macjeff said:

Works now.  I wanted to report just in case but there was a temp outage.

With that said it was still cached on the old info and did not change (I had admin console open) so I did a query on it and it all updated fine.  But shouldnt it have updated auto?

When the node server sends data that is different from the cached values, Polyglot is supposed to send that to the ISY.

I believe query (in this version) simply sends the cached values to the ISY.

Getting the data to the ISY is a two step process

1) the node server queries NOAA and sends the data to Polyglot

2) polyglot compares the data with cached values and if different sends it to the ISY

So short answer, is yes, when it queried new data, it should have updated the ISY automatically.

[cosyn]

Everything was going fine but today's wind event in the Bay Area caused a power outage. I was on battery backup for a bit with lots of brownouts but am back on grid. I noticed the WeatherPoly was no longer reporting. After a Polisy reboot, I got the following message

Failed to start weather monitoring service.

Here's an error message in the WP nodeserver log:

2021-10-11 14:39:53,892 MQTT       polyinterface      ERROR    polyinterface:_message: MQTT Received Unknown Error: An exception of type AttributeError occured. Arguments:
("'Controller' object has no attribute 'server'",)
Traceback (most recent call last):
  File "/var/polyglot/.local/lib/python3.8/site-packages/polyinterface/polyinterface.py", line 275, in _message
    self.stop()
  File "/var/polyglot/.local/lib/python3.8/site-packages/polyinterface/polyinterface.py", line 371, in stop
    watcher()
  File "./weatherstation.py", line 239, in stop
    self.server.Stop = True
AttributeError: 'Controller' object has no attribute 'server'

 

And in the Polisy log:

10/11/2021, 14:39:58 [polyglot] info: NS removenotice: WeatherPoly removed notice sucessfully.
10/11/2021, 14:39:59 [polyglot] error: removenode: Error: Node not found.
    at Object.removenode (/snapshot/polyglot-v2/lib/modules/nodeserver.js:472:30)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
10/11/2021, 14:39:59 [polyglot] info: NS addnotice: WeatherPoly added notice sucessfully: {"key":null,"value":"Failed to start weather monitoring service."}
10/11/2021, 14:39:59 [polyglot] error: removenode: Error: Node not found.
    at Object.removenode (/snapshot/polyglot-v2/lib/modules/nodeserver.js:472:30)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

 

@bpwwer, let me know if there's anything I should try before I resort to deleting and re-adding the NS. Thanks in advance.

 

[bpwwer]

13 minutes ago, cosyn said:

Everything was going fine but today's wind event in the Bay Area caused a power outage. I was on battery backup for a bit with lots of brownouts but am back on grid. I noticed the WeatherPoly was no longer reporting. After a Polisy reboot, I got the following message

Failed to start weather monitoring service.

@bpwwer, let me know if there's anything I should try before I resort to deleting and re-adding the NS. Thanks in advance.

 

Have you tried stopping and restarting the node server?  

The node server is running a small web server and that's how it gets data from whatever software you're using.  For some reason, it was unable to start the web server.  Typically, this is because something else is has control of the IP address/port combination.  But without more information on the error, that's just a guess.

If you enable debug level logging, it should report a more detailed error on why it failed to start the web server.

The error in the log looks like it is just complaining because it was unable to stop the web server when stopping the node server (which is expected since the web server never started). 

I don't believe that deleting and re-adding will help, at least not yet.

[cosyn]

1 hour ago, bpwwer said:

Have you tried stopping and restarting the node server?  

If you enable debug level logging, it should report a more detailed error on why it failed to start the web server.

The error in the log looks like it is just

I tried stopping/restarting a couple times and just tried again--same error. How do I change the logging level? Thanks for the responsiveness!

[bpwwer]

59 minutes ago, cosyn said:

I tried stopping/restarting a couple times and just tried again--same error. How do I change the logging level? Thanks for the responsiveness!

There should be a control on the main node in the admin console,  I think that should still work even though the web server isn't starting.

[cosyn]

2 hours ago, bpwwer said:

There should be a control on the main node in the admin console,  I think that should still work even though the web server isn't starting.

That's strange, I don't have a log control there. Here's what I see (also opened the NS config just in case it was there). In case you were looking for the info and other log messages, here's the complete log from the last restart (I only copied in the error message before).

 

2021-10-11 17:15:44,677 MQTT       polyinterface      ERROR    polyinterface:_message: MQTT Received Unknown Error: An exception of type AttributeError occured. Arguments:
("'Controller' object has no attribute 'server'",)
Traceback (most recent call last):
  File "/var/polyglot/.local/lib/python3.8/site-packages/polyinterface/polyinterface.py", line 275, in _message
    self.stop()
  File "/var/polyglot/.local/lib/python3.8/site-packages/polyinterface/polyinterface.py", line 371, in stop
    watcher()
  File "./weatherstation.py", line 239, in stop
    self.server.Stop = True
AttributeError: 'Controller' object has no attribute 'server'
2021-10-11 17:15:48,235 MainThread polyinterface      INFO     polylogger:set_basic_config: set_basic_config: enable=True level=30
2021-10-11 17:15:49,318 MainThread polyinterface      INFO     polyinterface:init_interface: Received Config from STDIN.
2021-10-11 17:15:49,319 MainThread polyinterface      INFO     __init__:<module>: UDI Polyglot v2 Interface 2.1.0 Starting...
2021-10-11 17:15:49,368 MainThread polyinterface      DEBUG    polyinterface:get_network_interface: gws: {'default': {2: ('xxx.xxx.xxx.1', 'igb0')}, 2: [('xxx.xxx.xxx.1', 'igb0', True)]}
2021-10-11 17:15:49,368 MainThread polyinterface      DEBUG    polyinterface:get_network_interface: gw: default=('xxx.xxx.xxx.1', 'igb0')
2021-10-11 17:15:49,370 MainThread polyinterface      DEBUG    polyinterface:get_network_interface: ifad: igb0=[{'addr': 'xxx.xxx.xxx.106', 'netmask': '255.255.255.0', 'broadcast': 'xxx.xxx.xxx.255'}]
2021-10-11 17:15:49,370 MainThread polyinterface      INFO     polyinterface:__init__: Connect: Network Interface: {'addr': 'xxx.xxx.xxx.106', 'netmask': '255.255.255.0', 'broadcast': 'xxx.xxx.xxx.255'}
2021-10-11 17:15:49,372 Interface  polyinterface      INFO     polyinterface:_startMqtt: Connecting to MQTT... localhost:1883
2021-10-11 17:15:49,374 MainThread polyinterface      INFO     weatherstation:<module>: Calling Controller to create controller node
2021-10-11 17:15:49,421 MQTT       polyinterface      INFO     polyinterface:_connect: MQTT Connected with result code 0 (Success)
2021-10-11 17:15:49,423 MQTT       polyinterface      INFO     polyinterface:_connect: MQTT Subscribing to topic: udi/polyglot/ns/6 -  MID: 1 Result: 0
2021-10-11 17:15:49,423 MQTT       polyinterface      INFO     polyinterface:_connect: MQTT Subscribing to topic: udi/polyglot/connections/polyglot -  MID: 2 Result: 0
2021-10-11 17:15:49,424 MQTT       polyinterface      INFO     polyinterface:_connect: Sent Connected message to Polyglot
2021-10-11 17:15:49,429 MQTT       polyinterface      INFO     polyinterface:_subscribe: MQTT Subscribed Succesfully for Message ID: 1 - QoS: (0,)
2021-10-11 17:15:49,433 MQTT       polyinterface      INFO     polyinterface:_subscribe: MQTT Subscribed Succesfully for Message ID: 2 - QoS: (0,)
2021-10-11 17:15:49,517 NodeServer polyinterface      INFO     weatherstation:start: Starting WeatherPoly Node Server
2021-10-11 17:15:49,519 NodeServer polyinterface      INFO     polyinterface:saveCustomData: Sending customData to Polyglot.
2021-10-11 17:15:49,803 NodeServer polyinterface      INFO     weatherstation:set_logging_level: set_logging_level: Setting log level to 0

 

[bpwwer]

Strange, I just installed it and I'm not seeing the log level control either.  I don't know why it's not showing up.  Looks like it is setting the log level to "off" which is why the log message stop after it sets the log level.

I'll have to look into it more to figure out what's going on.

[bpwwer]

I just pushed version 0.1.12

Please update to this version. It should solve the logging control issue and it should report the error if/when the web server fails to start.

[cosyn]

4 hours ago, bpwwer said:

I just pushed version 0.1.12

Please update to this version. It should solve the logging control issue and it should report the error if/when the web server fails to start.

Thanks. I tried to update but got this error:

 

 The last time this happened during repeated failed updates, I submitted a support ticket and @Michel Kohanim remoted into my computer and did a lot of SSH magic. Seemed like there was some sort of corruption in the system again. I'll go ahead and submit another ticket. Thanks again.

Edited October 12, 2021 by cosyn
removed duplicate image

[cosyn]

1 hour ago, cosyn said:

 The last time this happened during repeated failed updates, I submitted a support ticket and @Michel Kohanim remoted into my computer and did a lot of SSH magic. Seemed like there was some sort of corruption in the system again. I'll go ahead and submit another ticket. Thanks again.

 

Michel fixed it. Some sort of synch error between my Polisy and GitHub stopping updates. Whatever else he did, the weather server in the NS went back online. We agreed I'm producing too many EMPs making advanced hardware stop working. Matrix 4, here we come!

Edited October 12, 2021 by cosyn

[oneklakes]

7 minutes ago, simplextech said:

Please do check with him as he has been supporting the weather services much more than I have been.  He and I discussed a while ago about combining them and reducing duplicates/confusion of certain nodeservers to simplify options and support.

Hi Bob,

I'm transitioning over to your NS per the suggestion above. Simplextech identified an issue in the log file for his NS that was causing no data to be outputted for any of my temperature/humidity sensors even though I could see the output in the Ambient Dashboard.

I installed your NS (v. 0.1.11 running on firmware 5.3.3) and am still having the same issue. I looked at the log file (attached) and am not seeing the same error message so am not sure what is going on. I would appreciate it if you can take a look. Thanks!

 

ambientpoly-0021b90243c7-12.txt

[bpwwer]

@oneklakesIt looks like the node server is getting data but it doesn't recognize anything but pressure.  Whatever sensor(s) you have don't appear to be supported by the my node server.  Either that or Ambient weather has changed the format of the data.

I'm not enhancing any of my PG2/PGC node servers as my focus has shifted to PG3 versions. 

[oneklakes]

12 minutes ago, bpwwer said:

@oneklakesIt looks like the node server is getting data but it doesn't recognize anything but pressure.  Whatever sensor(s) you have don't appear to be supported by the my node server.  Either that or Ambient weather has changed the format of the data.

I'm not enhancing any of my PG2/PGC node servers as my focus has shifted to PG3 versions. 

@bpwwerThe sensors were the original ones that I was using with Simplextech's NS and were working at least a couple of months ago. One of those sensors died and was replaced and that's when things stopped working. Do you think generating a new API key would make a difference?

I understand that you are focusing on PG3 at this point so I don't expect you to make any changes to your PGC NS. Thanks for taking a look and getting back to me.

[simplextech]

7 minutes ago, oneklakes said:

One of those sensors died and was replaced and that's when things stopped working. Do you think generating a new API key would make a difference?

No the new API key would not cause this.

I'm in the same position as @bpwwer of not enhancing or changing PG2/PGC nodeservers and I plan to deprecate/remove them as I migrate them to PG3 as well.

I will look into this issue with Ambient Weather version of the NS and set it up on my test environment again.  If the problem is easy to identify and fix then I will.  I have moved away from Ambient Weather devices and I now recommend the EcoWitt GW1000 gateway for receiving and accessing PWS and Sensor data as it's all local.  However this does require a Polisy to be running Polyglot.