Weatherbug datafeed no response

[johnny66]

I'm running 3.3.10 on an ISY99i. For the last two weeks Weatherbug hasn't been working for me. It repeatedly shows in the event viewer "No response from datafeed.weatherbug.com."

 

Name resolution and NTP are working OK so internet access isn't the problem here.

 

On my router I did a packet capture of the unit's Weatherbug data requests. See attached (change the txt file extension to cap). It's requesting URL:

http://datafeed.weatherbug.com/GetXml.a ... UnitType=0

 

The packet capture shows that in response to the request the ISY only receives a SYN-ACK back from the server at datafeed.weatherbug.com, it doesn't receive data back from the Weatherbug server. The ISY then waits for a while, then makes the same request. Over and over again. It never receives data back.

 

However, when I request the same URL in a browser on my PC, the request succeeds and returns the expected XML fully populated with the expected weather data.

 

I've tried clearing my Java cache, choosing different nearby stations, rebooting the ISY, etc. This doesn't help, I observe the same behavior.

 

What should I try next?

130526-1445_ISY-to-Internet_packetcapture.txt

[Michel Kohanim]

Hi johnny66,

 

This is really very odd. Would you be kind enough to:

1. Reboot ISY, retry ... any improvements?

2. Reboot router, retry ... any improvements?

 

With kind regards,

Michel

[johnny66]

Thanks Michael for your suggestions. I power cycled the ISY and rebooted my router. But this doesn't appear to have helped, the ISY is still failing to retrieve Weatherbug data.

 

I just noticed that the last date that Weatherbug worked for me, 5/12, was right before I upgraded from 3.2.6 to 3.3.10, 5/13. Previous to this Weathbug had been reliable for months. It looks like the misbehavior is associated with this upgrade. Perhaps I should revert back to 3.2.6?

 

This misbehavior is a bit painful for us because we're running EZFlora units with 10 irrigation zones and the program I'm using relies on Weatherbug data.

[johnny66]

Inspecting the packet capture of the Weatherbug request from the ISY (doesn't work) versus the PC (works), it appears that the request from the ISY is malformed. The request from the ISY has a space between the hostname (datafeed.weatherbug.com:80) and the URI (/GetXml.aspx?...). Most webservers disallow spaces and consider a request containing spaces to be malformed. So the Weatherbug webserver is unable to parse the request from the ISY.

 

See attached packet capture which contains Weatherbug requests from both ISY (10.1.130.125) and PC (10.1.130.113).

 

From ISY:

http://datafeed.weatherbug.com:80 /GetXml.aspx?RequestType=3&PartnerId=3faff2bf-c622-41bd-9d46-f1a23cd089d0&StationId=LSNGL&UnitType=0

 

From PC:

http://datafeed.weatherbug.com/GetXml.a ... UnitType=0

 

EDIT -- I just downgraded to 3.2.6 but unfortunately I'm observing the same behavior. Requests to Weatherbug still failing.

130527-0050_ISY-to-Internet_packetcapture.cap.txt

[Michel Kohanim]

Hi johnny66,

 

Thank you. HTTP commands do not have URLS. They have headers, so it would be:

GET /whatever-path HTTP/1.1

Host: whatever-host

...

 

As such, what you see is the tool interpreting HTTP headers. I really do not think that's the issue. It would be really great if you could:

1. Backup ISY

2. Downgrade to 3.2.6 and let me know if that fixes the issue

 

With kind regards,

Michel

[johnny66]

Michael, I downgraded to 3.2.6 but continue to observe the same behavior.

 

Did you look at the packet capture? This is the data on the wire. To me, looking at the packet capture in Wireshark, it's readily apparent why the request from the ISY is failing: because the request contains an illegal un-encoded space. When a web browser requests a URL containing a space it replaces these with characters which represent or encode the space, "%20". But the ISY is failing to do that.

 

The packet capture shows a PC making a request a minute after the ISY's failed request. The request from the PC is completely identical to the ISY's request except it doesn't contain a space, and it succeeds.

 

As to the question of WHY my ISY is making this malformed request, that I don't know. The host and URI for Weatherbug aren't exposed in the ISY GUI, so there's no way for a user to modify them. But perhaps they're corrupted.

 

 

Some references to spaces in URLs

 

http://en.wikipedia.org/wiki/Query_string

Some characters cannot be part of a URL (for example, the space)... In particular, encoding the query string uses the following rules: SPACE is encoded as '+' or "%20".

 

http://tools.ietf.org/html/rfc1738

Characters can be unsafe for a number of reasons. The space character is unsafe because significant spaces may disappear and insignificant spaces may be introduced when URLs are transcribed or typeset... All unsafe characters must always be encoded within a URL.

 

http://stackoverflow.com/questions/5442 ... es-in-urls

A URL must not contain a literal space. It must either be encoded using the percent-encoding or a different encoding that uses URL-safe characters... Syntactically, a URI must not contain a literal space and it must be encoded; semantically, a %20 is not a space (obviously) but it represents a space.

[Michel Kohanim]

Hello johnny66,

 

As I mentioned before, the URL you see on your tool is NOT what's on the wire. This, I guarantee you.

 

Whatever you type on the browser is converted to something like:

GET /path-to-whatever HTTP/1.1

Host: whatever-host

 

Multiple spaces between GET and path should be tolerated. And, we have 1000s of these modules installed and none exhibit the same behavior.

 

And, finally, nothing has changed since 3.2.6 and you still have the same issue with 3.2.6. So, it's very hard for me to accept that ISY is the issue here.

 

Has anything else changed? Perhaps a new router?

 

How long have you had your ISY and what model is it?

 

With kind regards,

Michel