Polyglot weather station node servers

[macjeff]

Strange issue with NOAA Warnings-  

The other day I ignored it but we had a Winter Storm Warning and it said Tsunami Warning.  Big difference  LOL

Today there is a Winter Weather Advisory and it says NONE.

See screenshots.

 

[gviliunas]

@macjeffIn may part of the world, Weather Forecaster is the only job where you can be right 20% of the time and still get a big salary increase at the end of the year. 

I hope that your Tsunami was "Moderate" or lower.  Also, "Winter Weather Advisory" is just not sexy in February..... LOL

[bpwwer]

2 hours ago, macjeff said:

Strange issue with NOAA Warnings-  

The other day I ignored it but we had a Winter Storm Warning and it said Tsunami Warning.  Big difference  LOL

Today there is a Winter Weather Advisory and it says NONE.

See screenshots.

I had a typo in the id used for Winter Storm Warning, it was the same id as Tsunami Warning.  I've fixed that in version 1.1.1 that should be available in the store in less than an hour.

When I made the node server, NOAA didn't define anything called Winter Weather Advisory so it doesn't match and thus, is NONE.  This is the main problem with NOAA forecasts and alerts.  They use text strings and to work with that, the node server has to be able to match those strings exactly but apparently they don't define what all the strings are that they use.

Since the ISY doesn't support string data, strings have to be mapped to numbers and I can only do that if I know what the strings are going to be.

[MrBill]

On 2/10/2021 at 11:27 AM, bpwwer said:

I had a typo in the id used for Winter Storm Warning, it was the same id as Tsunami Warning.  I've fixed that in version 1.1.1 that should be available in the store in less than an hour.

When I made the node server, NOAA didn't define anything called Winter Weather Advisory so it doesn't match and thus, is NONE.  This is the main problem with NOAA forecasts and alerts.  They use text strings and to work with that, the node server has to be able to match those strings exactly but apparently they don't define what all the strings are that they use.

Since the ISY doesn't support string data, strings have to be mapped to numbers and I can only do that if I know what the strings are going to be.

I'm also pretty certain "advisory" is neither a "watch" nor "warning".   In other words it's a 3rd  class. https://www.weather.gov/sjt/WatchWarningAdvisoryExplained

[DennisC]

@bpwwer

I'm seeing the following error in the log for NOAA:

2021-02-15 08:26:50,373 NodeServer polyinterface      ERROR    polyinterface:write: Exception in thread NodeServer:

Traceback (most recent call last):

  File "/var/polyglot/nodeservers/NOAA/nodes/noaa.py", line 175, in query_alerts

    noaa = ET.fromstring(xdata)

  File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1315, in XML

    parser.feed(text)

  File "<string>", line None

xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):

  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner

    self.run()

  File "/usr/local/lib/python3.7/threading.py", line 870, in run

    self._target(*self._args, **self._kwargs)

  File "/var/polyglot/nodeservers/NOAA/nodes/noaa.py", line 86, in start

    self.query_alerts(self.params.get('Alert zone/county code'))

  File "/var/polyglot/nodeservers/NOAA/nodes/noaa.py", line 221, in query_alerts

    except exception as e:

NameError: name 'exception' is not defined

2021-02-15 08:36:49,077 Controller polyinterface      DEBUG    noaa:longPoll: longpoll

2021-02-15 08:36:49,596 Controller polyinterface      DEBUG    noaa:query_alerts: Parse XML and set drivers

2021-02-15 08:36:49,603 Controller polyinterface      ERROR    polyinterface:write: Exception in thread Controller:

Traceback (most recent call last):

  File "/var/polyglot/nodeservers/NOAA/nodes/noaa.py", line 175, in query_alerts

    noaa = ET.fromstring(xdata)

  File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1315, in XML

    parser.feed(text)

  File "<string>", line None

xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):

  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner

    self.run()

  File "/usr/local/lib/python3.7/threading.py", line 870, in run

    self._target(*self._args, **self._kwargs)

  File "/var/polyglot/.local/lib/python3.7/site-packages/polyinterface/polyinterface.py", line 854, in _parseInput

    self.longPoll()

  File "/var/polyglot/nodeservers/NOAA/nodes/noaa.py", line 91, in longPoll

    self.query_alerts(self.params.get('Alert zone/county code'))

  File "/var/polyglot/nodeservers/NOAA/nodes/noaa.py", line 221, in query_alerts

    except exception as e:

NameError: name 'exception' is not defined

 

Also, for the Alert zone/county code are you looking for both values like this: xxxxxx/xxxxxx?

Thanks.

Edited February 15, 2021 by DennisC
Updated error

[bpwwer]

4 hours ago, DennisC said:

@bpwwer

I'm seeing the following error in the log for NOAA:

Also, for the Alert zone/county code are you looking for both values like this: xxxxxx/xxxxxx?

Thanks.

Thanks for the report.  I think I have that fixed now.

It's looking for just the single 6 character code. It can be either an zone code or a county code.  For example " AZC001" for Apache county AZ.

[macjeff]

5 hours ago, bpwwer said:

Thanks for the report.  I think I have that fixed now.

It's looking for just the single 6 character code. It can be either an zone code or a county code.  For example " AZC001" for Apache county AZ.

Not trying to push but looked for this update and dont see it.  When will you post?

[Bumbershoot]

@bpwwer, a couple of development requests to the UDP WeatherFlow nodeserver for when time permits. 

I'm sure you're aware that the WeatherFlow Forecast API is now live, and it looks to provide way too much data (10 daily forecasts, 240 hourly forecasts, etc.).  It would, however, be nice to have the WeatherFlow nodeserver match forecast parity with the other weather related nodeservers that you've so generously written.

I've now gotten multiples of WeatherFlow hardware available -- I currently have a Tempest in production and a spare Air unit that I'd like to use inside, if my ISY could make productive use of temperature and humidity data from both the Tempest and the Air.  Currently, while on the same network, the Tempest and Air temperature, humidity, barometric pressure, etc., share nodes in nodeserver, meaning the those node values alternate between Tempest and Air (or inside and outside).  I've worked around this by putting another hub on a different LANs and by using the UDP nodeserver for the Tempest and PGC nodeserver for the Air.  Kludgey, but it works.  Better would be to add the AIR to the same hub as the Tempest and have the values go to different nodeserver nodes.

I appreciate very much your continual work on these nodeservers.

[macjeff]

 

8 minutes ago, Bumbershoot said:

@bpwwer, a couple of development requests to the UDP WeatherFlow nodeserver for when time permits. 

I'm sure you're aware that the WeatherFlow Forecast API is now live, and it looks to provide way too much data (10 daily forecasts, 240 hourly forecasts, etc.).  It would, however, be nice to have the WeatherFlow nodeserver match forecast parity with the other weather related nodeservers that you've so generously written.

I've now gotten multiples of WeatherFlow hardware available -- I currently have a Tempest in production and a spare Air unit that I'd like to use inside, if my ISY could make productive use of temperature and humidity data from both the Tempest and the Air.  Currently, while on the same network, the Tempest and Air temperature, humidity, barometric pressure, etc., share nodes in nodeserver, meaning the those node values alternate between Tempest and Air (or inside and outside).  I've worked around this by putting another hub on a different LANs and by using the UDP nodeserver for the Tempest and PGC nodeserver for the Air.  Kludgey, but it works.  Better would be to add the AIR to the same hub as the Tempest and have the values go to different nodeserver nodes.

I appreciate very much your continual work on these nodeservers.

Fyi-  I have beta tested every unit for them.  I use this nodeserver but also run Nodelink.  Nodelink can have multiple instances of weatherflow but not tempest.  So I run Tempest in Polisy using this nodeserver and multiple nodelink weatherflow devices.  There is a raspberry pi script that installs everything with one click.

https://forum.universal-devices.com/forum/113-nodelink/

With that said I know its in the plans once Polyglot 3 is out to add multiple weatherflows.  

Also I SECOND the request for Forecast especially with Dark Sky pretty much gone.  I use Weatherbit, openweathermap, and NOAA now.

[Bumbershoot]

11 minutes ago, macjeff said:

There is a raspberry pi script that installs everything with one click.

Thanks for that.  I used to run WeatherFlow on NodeLink, but I'm not running .NET on my RPi currently, as it was repurposed for another task.  I haven't looked into putting .NET on my Polisy, though it wouldn't run in the past.  I haven't looked into it recently.

[macjeff]

11 minutes ago, Bumbershoot said:

Thanks for that.  I used to run WeatherFlow on NodeLink, but I'm not running .NET on my RPi currently, as it was repurposed for another task.  I haven't looked into putting .NET on my Polisy, though it wouldn't run in the past.  I haven't looked into it recently.

There was a script install the latest node link And also installs.net so it makes it simple and you can run it with other stuff which I do

[bpwwer]

Support for the WeatherFlow forecast will be in the next release, that's mostly coded at this point. 

Support for multiple devices/hubs is a much larger/complex feature.  The original stand-alone node server I wrote had that support and, at the time, the Polyglot based version was supposed to be a more simple version.  That's obviously changed over time and the Polyglot version needs to evolve.

[macjeff]

Bob,

I mentioned this before and you said you would put on the end of your list due to PG3. 

My issue happened again this AM.   For some reason all of my nodeservers reported down (online=FALSE).  Probably internet related.  My program now waits 30 min and then sends a query to all the nodesevers.  It then enables and runs a second program which re-evaluates.  If Still FALSE it sends a text to me.  Then it waits an hour before checking again.

In all of my nodeservers (every one now except your weather ones) the query worked and the programs never sent me the text.  EXCEPT all of your nodeservers.  The programs for your weather nodeservers were still in the dark green icon showing running and had sent the text (they were waiting the hour).  So all I did is go into admin and do a QUERY on the node server (and it was showing True- online already) and the programs instantly stopped running.   So the query command in the program is the key (not sure why it cant figure out on its own since there is a status change!!!)

Anyways, you said it was a major deal to do that.   There were a few other nodeservers that did not have Query and I asked them to add and they did the same day.  One was total connect and a couple people put thank yous on the forum for adding it.  It just has to do the same command as if you right clicked on the nodeserver and chose Query.

Not trying to push but want to make sure you dont forget this on your list. I use a few of yours (NOAA, Climacell, Openweathermap. Weatherbit, and Weatherflow) but I am sure others would like to see it in all of them (well I would not mess with Dark Sky at this point)!!

If you have any other suggestions let me know.  I tried some of the other commands you do offer in programs like Rediscover, etc.

thanks!!

[bpwwer]

@macjeff, the problem I have with this is that I don't like to do the wrong thing to fix something that shouldn't be a problem in the first place.

The query request from the ISY to a node server is supposed to do a specific action.  It is asking the node server to get the very latest data from the device and report it.   The node server framework has a base implementation of this that reports the current values.  This is just a placeholder since it doesn't know how to query what device(s) the node server is talking to.

It's very easy to expose that placeholder as is.  This is what most (maybe all) of the node servers are doing, but this wrong.

The correct way to implement this for the weather node servers is to query the services for the most recent data and report that back to the ISY.  Thus the correct implementation is a bit more involved than just exposing the placeholder.

However, all your node servers shouldn't be going off-line.  That's really the problem you need to fix.  The fact that you've found a way to work-around it is commendable, but you really should figure out why they're going off-line. That sounds like a problem that's only going to get worse if it's not resolved.

[macjeff]

29 minutes ago, bpwwer said:

@macjeff, the problem I have with this is that I don't like to do the wrong thing to fix something that shouldn't be a problem in the first place.

The query request from the ISY to a node server is supposed to do a specific action.  It is asking the node server to get the very latest data from the device and report it.   The node server framework has a base implementation of this that reports the current values.  This is just a placeholder since it doesn't know how to query what device(s) the node server is talking to.

It's very easy to expose that placeholder as is.  This is what most (maybe all) of the node servers are doing, but this wrong.

The correct way to implement this for the weather node servers is to query the services for the most recent data and report that back to the ISY.  Thus the correct implementation is a bit more involved than just exposing the placeholder.

However, all your node servers shouldn't be going off-line.  That's really the problem you need to fix.  The fact that you've found a way to work-around it is commendable, but you really should figure out why they're going off-line. That sounds like a problem that's only going to get worse if it's not resolved.

I live in the country and we have frequent power outages for a few seconds and Internet outages. I got around the power outages by putting almost everything on battery back ups including my Internet connection but it does go down on its own. And for some reason around four or five in the morning on a weekly basis we lose Internet for about 20 minutes or so which is enough to send a false Nodeserver. That’s the only thing I can think of since it always seems to happen while I’m sleeping and it comes back on its own and only about once a week. It could be a quick power outage. Maybe I should put my polisy on a UPS which it’s not 

[bpwwer]

2 hours ago, macjeff said:

I live in the country and we have frequent power outages for a few seconds and Internet outages. I got around the power outages by putting almost everything on battery back ups including my Internet connection but it does go down on its own. And for some reason around four or five in the morning on a weekly basis we lose Internet for about 20 minutes or so which is enough to send a false Nodeserver. That’s the only thing I can think of since it always seems to happen while I’m sleeping and it comes back on its own and only about once a week. It could be a quick power outage. Maybe I should put my polisy on a UPS which it’s not 

Internet going out shouldn't effect the node server connection to Polyglot.  Both the node server and the Polyglot are running on the Polisy and that internal connection should be very stable.  If the Polisy isn't loosing power, then the connection should be stable, if the Polisy is loosing power then it seems like everything should re-start when power is restored.

An external Internet outage shouldn't have any effect unless it is also causing your local network to fail such that all traffic is blocked.  

I still think it would be best to try and understand what's causing the node servers to go off-line, but I'll try to make some time to add the query command.

 

[simplextech]

6 hours ago, macjeff said:

Maybe I should put my polisy on a UPS which it’s not 

This would be the very first thing I would do.

[macjeff]

Bob,

There was an Air Quality Alert out here but its not showing up in NOAA nodeserver.  Is that one you ignore?

 

Thanks!!

 

 

 

 

[bpwwer]

16 hours ago, macjeff said:

Bob,

There was an Air Quality Alert out here but its not showing up in NOAA nodeserver.  Is that one you ignore?

I'm not ignoring anything, but because the ISY only works with numeric values and not random strings I can only process events that are documented.  I got the list of possible events from https://www.weather.gov/nwr/eventcodes and that doesn't include anything for air quality.

[macjeff]

3 hours ago, bpwwer said:

I'm not ignoring anything, but because the ISY only works with numeric values and not random strings I can only process events that are documented.  I got the list of possible events from https://www.weather.gov/nwr/eventcodes and that doesn't include anything for air quality.

Its ok.  Air Quality alerts are stupid anyways.  Whenever it gets hot and humid they issue them.  

Just reporting in case its s bug!!

THANKS!!

[bpwwer]

Thanks for reporting it!  I get annoyed when things are documented well because then I have to try and defend why it may not be working as user's expect.  Not annoyed at you, at NOAA.

If you're concerned about outdoor air quality, look into Purple Air.  There are a lot people (and organizations) that have deployed them and many have their data accessible to the public.  Using my Purple Air node server you can access that data and generate your own alerts based on a nearby sensor.  Or buy your own, that's what I did because we've had a lot of smokey days from the CA wildfires.

[DaveStLou]

2 hours ago, bpwwer said:

If you're concerned about outdoor air quality, look into Purple Air.  There are a lot people (and organizations) that have deployed them and many have their data accessible to the public.  Using my Purple Air node server you can access that data and generate your own alerts based on a nearby sensor.  Or buy your own, that's what I did because we've had a lot of smokey days from the CA wildfires.

@macjeffAnother option is to use World Air Quality Index project (AGICN.org) which is one I use on my weather page (OakvilleWX.com).

This link will take you to the station in their database nearest to you: https://aqicn.org/here/

They also have an API although I can't think of a use case in ISY, someone else may have one.

[Bumbershoot]

21 hours ago, macjeff said:

Air Quality alerts are stupid anyways.

I used the Purple Air nodeserver last fall when the Oregon wildfires were raging out-of-control, and I was pleased with it.  I did, however, find that the closest sensor to me was made unreliable because (I think) the owner kept taking the sensor indoors while the outdoor air was terrible.  I tried to buy one of my own, but there was a months long lead time, so I bought one of these Ecowitt sensors (it was available on Amazon at the time).  It provides what I need, it's solar powered, and there's a node server (EcoWitt) in the store that recognizes it, so you can use local outdoor AQI values in your ISY.  These are nowhere nearly as widely deployed at the Purple Air sensors, and as a result, the map display of the sensor network isn't even close to being as useful as the Purple Air map.  The only negative comment I'll make about the device is that it seems to go offline occasionally, but I think that's to save battery when the data doesn't change.  It always seems to come back to life.

If you're going to buy a AQI sensor, get one on the way prior to fire season.

Any port in a storm...

[macjeff]

There is a purpleair device right near me.  Any way to get its ID so I can monitor that one?  I found the map but dont see the ID's listed?  Otherwise I would have to buy one.

 

[bpwwer]

2 hours ago, macjeff said:

There is a purpleair device right near me.  Any way to get its ID so I can monitor that one?  I found the map but dont see the ID's listed?  Otherwise I would have to buy one.

 

Yes.  Click on the monitor and it brings up the green box with sensor information.  At the bottom of that box is a link to "Get this widget".  If you hover over that link, there will be an ID with the sensor number shown.   For example : 

<div id='PurpleAirWidget_68719_module_AQI_conversion_C0_average_10_layer_standard'>Loading PurpleAir Widget...</div>
<script src='https://www.purpleair.com/pa.widget.js?key=RRFYHSR646VW2YIQ&module=AQI&conversion=C0&average=10&layer=standard&container=PurpleAirWidget_68719_module_AQI_conversion_C0_average_10_layer_standard'></script>

The sensor ID is the 68719.