Node Server for NOAA XML weather data

[Craigb]

I'm searching for a weather node server that can scrape data from NOAA's XML data feeds. 

Example:  https://w1.weather.gov/xml/current_obs/KMDW.xml

These provide basic metar type data, and other links based on observation site can provide 3 day history as well.
 

Is it possible to configure WeatherPoly to retrieve the data from NOAA?  I couldn't figure out how to get WeatherPoly to poll the website.

Edited August 16, 2020 by Craigb

[bpwwer]

WeatherPoly doesn't work that way, it runs a server and expects data to be pushed to it, it doesn't poll.

A basic node server to pull NOAA data wouldn't be too hard to create.  I'll look into it.

[bpwwer]

Not too hard.  There's now a NOAA node server in the store.

[DaveStLou]

On 8/16/2020 at 5:23 PM, bpwwer said:

Not too hard.  There's now a NOAA node server in the store.

Nice job, @bpwwer

I've included a screenshot for those who are curious about what information you get via this node server:

Edited August 19, 2020 by DaveStLou

[Craigb]

Wow, thanks!

[vbPhil]

I'm not getting some of the data. Was especially interested in Climate Conditions which I guess is the same as Weather Conditions as on the NOAA site. I believe it's what will say if it's raining or not.

Any ideas?

 

[bpwwer]

Unknown means that whatever they are sending, isn't something that was documented as a valid condition.  Some of the NOAA data is frustratingly difficult to deal with as they do a very poor job of documenting what all the possible conditions are. 

The ISY can only deal with fixed values so every condition needs to be pre-defined and mapped to a numeric value for use with the ISY.

To get the alert data you'd have to configure the node server with an alert area which it doesn't look like you did.

[vbPhil]

okay.

It's now showing "Overcast". Before when it was showing "Unknown" it was "Light Rain" at the NOAA web site.

I do have the Alert County Code configured.

 

[vbPhil]

I see where NOAA publishes a list of their possible Weather Conditions used in their XML data. It is quite lengthy.

https://w1.weather.gov/xml/current_obs/weather.php

[bpwwer]

Ah, so probably no alerts are available at this time for that region.

That looks like the same list I used.  About 269 different conditions.  It looks like there's an issue where "Light Rain" and "Drizzle" got combined as one condition when they should be two separate conditions.

[vbPhil]

I just checked how the NOAA node was looking from the UD Mobile iPhone App and it looks like it's displaying the index number instead of the value.

 

[DennisC]

13 hours ago, vbphil said:

I just checked how the NOAA node was looking from the UD Mobile iPhone App and it looks like it's displaying the index number instead of the value.

 

Try restarting your ISY and then resync ISY Mobile.

[vbPhil]

42 minutes ago, DennisC said:

Try restarting your ISY and then resync ISY Mobile.

That fixed it. Thanks

 

Edited June 30, 2021 by vbphil

[vbPhil]

16 hours ago, bpwwer said:

Ah, so probably no alerts are available at this time for that region.

That looks like the same list I used.  About 269 different conditions.  It looks like there's an issue where "Light Rain" and "Drizzle" got combined as one condition when they should be two separate conditions.

Is this issue on the Node Server's side or NOAA's data?

[bpwwer]

It was a problem with my translation from the web page to the ISY profile file.  I've fixed it already.

[vbPhil]

Kudos for fixing it.

[asbril]

I installed the PG3 NOAA nodeserver and it (apparently) added successfully on my ISY

During the installation it gave me a message that the short poll (3600) could not be higher than the long poll (600), and I changed it to  100 / 600.  Are these the apporpriate settings ?

 

[bpwwer]

1 hour ago, asbril said:

I installed the PG3 NOAA nodeserver and it (apparently) added successfully on my ISY

During the installation it gave me a message that the short poll (3600) could not be higher than the long poll (600), and I changed it to  100 / 600.  Are these the apporpriate settings ?

 

Depends on how you plan to use it. shortpoll is how often it will check for updated weather information, longpoll is how often it checks for alerts.

If you're planning to use alerts, how long can you wait for them?  

Now that I think about it, maybe I should switch longpoll and shortpoll as you are more likely to want more frequent checks for alerts.

[asbril]

1 hour ago, bpwwer said:

you are more likely to want more frequent checks for alerts.

yes ?

[asbril]

8 hours ago, bpwwer said:

Depends on how you plan to use it. shortpoll is how often it will check for updated weather information, longpoll is how often it checks for alerts.

If you're planning to use alerts, how long can you wait for them?  

Now that I think about it, maybe I should switch longpoll and shortpoll as you are more likely to want more frequent checks for alerts.

This brings me to another question, which is  the general purpose of short and long polls in nodeservers. Do the numbers for these polls in nodeserver configuration represent sec?onds 

In my ISY logs I have noticed that some nodeservers that I use (example Holidays Google) very frequently polls data while in fact that is not needed with my programs.  What numbers should I use if I prefer less frequent polling ?

[bpwwer]

3 hours ago, asbril said:

This brings me to another question, which is  the general purpose of short and long polls in nodeservers. Do the numbers for these polls in nodeserver configuration represent sec?onds 

In my ISY logs I have noticed that some nodeservers that I use (example Holidays Google) very frequently polls data while in fact that is not needed with my programs.  What numbers should I use if I prefer less frequent polling ?

Polyglot provides two timers that can periodically trigger actions in node servers.  How those are used in node servers is node server specific.   It's really just a helper capability since many node servers need to do something at a fixed interval. 

The numbers are in seconds.  

Typically, the node server will set some default values based on what they need to do periodically, you can modify them to make the times longer or shorter depending on your needs.  In some cases, there are limits or recomended limits to prevent the node server from overloading the service it is polling.   For example, many weather services have limit on the number times you can poll them per minute/per day/per month so you don't want to exceed that or you'll stop getting data. Others may be sending a lot of data to the ISY so if they do that too frequently, it will slow down the ISY.

The bigger the number, the less frequent the poll.  If you're only looking at days in the Holidays Google node server, you could set the polling to happen only once a day, or maybe once an hour.

[asbril]

2 hours ago, bpwwer said:

If you're only looking at days in the Holidays Google node server, you could set the polling to happen only once a day, or maybe once an hour.

Thanks so much @bpwwer For once an hour, what numbersshould I use (sort and/or long poll) ?

[bpwwer]

3600 seconds is 1 hour.  But I don't know what Google Holidays does the short vs. long.  But long needs to be larger than short.  So if you want shortpoll of 3600, you need to set longpoll first to something bigger.

[MWareman]

@bpwwer

Been loving this node - thank you. Started integrating alerts - and it appears alerts are coming in as integers values rather than mapped to their meanings.

In the admin UI - they show correctly:


However - in UDI Mobile they show as integers:

I am noticing that when using NR substitutions, it's the integers that come thru. For instance, this pushover payload:

priority=-1&retry=30&expire=3600&sound=none&message=${sys.node.#.GV21:url}+(${sys.node.#.GV23:url})+severity+${sys.node.#.GV25:url}+and+urgency+${sys.node.#.GV26:url}+and+certaincy+${sys.node.#.GV27:url})

This morning resulted in:

Is it possible to take a look and fix the mappings in the poly?

Thank you!

Michael.

[bpwwer]

It's not the node server.

The node server provides a file to the ISY/IoP with the mappings.  Everything else either gets the mapping from the ISY/IoP or gets the files from the ISY/IoP.

The admin console gets the files from the ISY/IoP when it starts and it does it's own parsing of those files.  Thus, if the admin console is correct then the node server has sent valid/correct files to the ISY.

It seems like UDI Mobile expects the ISY to provide the "formatted" (I.E. pre-parsed) values and it's not doing so.  @Javican provide more details on how UDI Mobile works.

While the ISY/IoP can send the files to the AC, if it hasn't been rebooted since the files were transferred to the ISY/IoP, it may not parse them and thus send out just the "unformatted" values when reporting status.   This is the most common case.

There's a small possibility that some is wrong in the files that the AC can handle but the ISY/UD Mobile can't. But since the AC is the only tool node server developers have to check the correctness of the files, we assume that if it works there, the files are good.