Jump to content

IoP Reboot- PG3 needs reboot?


macjeff

Recommended Posts

Node server status is just the status of the MQTT link between PG3 and the node server.  It only changes when the node server is started or stopped.

Restarting the ISY, has no impact on this.  The ISY is clearing the status when restarted and unless the node server is stopped and then restarted, it won't send any updates to the ISY.   

Typically PG3 will try to minimize the traffic to the ISY so it won't send a status update unless the status has actually changed.  Since the status hasn't changed, no update is ever sent.

With PG2, the node server status was handled differently and what it reported was different. There could be other differences around when it would/wouldn't send updates as it was not trying to minimize the traffic like PG3 does.  Thus PG2 and PG3 are not expected to behave the same.

Link to comment
15 hours ago, macjeff said:

I just tried it again.  Reboot IoP and then all PG3 nodeservers have mostly blank info.   I hit query and nothing.  Reboot Pg3 and it all comes in.

What I was trying to say above was that in many cases, this is the expected behavior with the current PG3 design.

Link to comment
23 minutes ago, bpwwer said:

What I was trying to say above was that in many cases, this is the expected behavior with the current PG3 design.

So is there any way to automate this.  Sometimes I do a reboot and then walk away.  I guess I will have to remember to reboot PG3 every time

Link to comment
2 minutes ago, Michel Kohanim said:

This probably means that IoP starts much later than PG2 and PG3, and by that time, PG has given up till the next poll. We'll definitely take a look.

With kind regards,
Michel

When I do this most the time I’m just choosing restart from within IoP.  I didn’t know that also rebooted the other services.  

Link to comment

There are two different things being discussed above, but both may have the same root cause.

If the ISY is not available when PG3 starts, it shouldn't effect the ability for PG3 to start node servers.  If PG3 fails to start the node server, then that is probably a bug.  However, if the ISY is not available, then PG3 then all status updates to the ISY will fail.  Because PG3 typically only sends status updates when the status actually changes, if the ISY becomes available after PG3 has given up, it won't try to send updates again until the status changes again.

Link to comment
1 minute ago, JimboAutomates said:

The other issue being discussed is that a query of a nodeserver node doesn't always send back the status to the ISY if it thinks there is no change.  If you restart ISY and have Query on Restart enabled then all current node status should come back to ISY, but currently it does not.

Couldn’t have said it better myself.  

I would be fine to wait for next update if it could get the last update.  PG3 was not rebooted so should still have current data.  ISY was rebooted but a lot of fields are empty and even a manual query does not bring the data in.  

This causes programs to not run correctly or run when they should not.  

Yes a reboot of PG3 fixes the issue but hope this is not the solution  

 

Jeff

Link to comment

Ok after updating isy on polisy to 5.4.0 i left it alone and started seeing updates from my pg3 ecobee. I then did a query from isy and everything populated. So I guess that is how it worked and I didn’t realize before. Pg2 I left alone and haven’t seen it in isy yet   . 

Link to comment

I just did my upgrade and will leave things.  A few nodeservers are fine like WeatherFlow which is Pg3.  But look at the screenshots attached.  NOAA is especially bad.  PurpleAir is totally empty.

I am not doing it now but as before I would leave an hour or two and nothing so I restarted Pg3 and then they are all fine.

So it might be weather ones are slow to respond but I did a Query on NOAA and got an error.  I attached a screenshot of that also.

 

1556713350_ScreenShot2022-03-01at7_45_49AM.thumb.png.59e061f09ad1529a2119c368a56222de.png

 

Screen Shot 2022-03-01 at 7.49.03 AM.png

Screen Shot 2022-03-01 at 7.46.44 AM.png

Screen Shot 2022-03-01 at 7.46.03 AM.png

Screen Shot 2022-03-01 at 7.45.41 AM.png

Link to comment

First I did clear JAVA cache so thats not the issue. Waited 3.5 hours for update

** seems like blank fields dont fill in but the ones that were not blank are updating

I have a more complex system than most so maybe my issue is it needs more time before PG2 and Pg3 initializes the fields in the the ISY.  Maybe this can be a setting?

Using PurpleAir as example I went to Pg3 and looked at the DC node I sent picture of above and all the values are there but 30 min after update they are all still blank.  Even if it takes a while to come in there has to be a way to force the values to come in even if the nodeserver only updates once a day. (Query All should do that)   Like if you have a program to put a siren off if some value is not a certain amount then the siren might go off.  So I would rather have the last known value than no value.

I took a loot at logs and there are errors (see at bottom of post).  I restarted purple air (not all of Pg3) and it started up with NO ERRORS and it is now updating fine.  Nodeservers that are not working .

Most nodeservers seem to have these errors.  And the errors seem to be close on all the ones by the same author

PG2 same issue

Linktap-  Data was in fields but would not update and no commands worked.  Had to resatart this one.  Could not wait. Everything worked after restart

Hue-  Some fields missing

Netatmo-  Status says true but all other fields in modules blank

Ecowitt- Controller online field is blank.  Other fields are fine and updating

 

Pg3 nodeservers I am using and whats missing:

     Climacell- Only missing Nodeserver online field.  Others seem to be updating but blank ones stay blank

     Weatherbit- Only missing Nodeserver online field  Others seem to be updating but blank ones stay blank

     NOAA- missing 1/2 the fields  Others seem to be updating but blank ones stay blank

     OpenWeatherMap- Only missing Nodeserver online field   Others seem to be updating but blank ones stay blank

      Notification-  This one is strange.  Would not send ANY notifications but all fields were fine.  Restarted the nodeserver and then all of them sent.  I can confirm all notifications were working before the ISY was restarted.

     Wireless Tag- Perfect!! No issues

     Ecobee- Perfect!! No issues

     Weatherflow- Perfect!! No issues

     Kasa- Perfect!! No issues

     Harmony Hub- Seems fine

     ST Inventory- Perfect!! No issues

     Timedata- Perfect!! No issues

     Purple Air- TOTALLY BLANK until restart as noted above and error log below

2022-03-01 07:39:18,214 Command udi_interface ERROR udi_interface:write: Exception in thread 2022-03-01 07:39:18,219 Command udi_interface ERROR udi_interface:write: Command 2022-03-01 07:39:18,219 Command udi_interface ERROR udi_interface:write: : 2022-03-01 07:39:18,220 Command udi_interface ERROR udi_interface:write: Traceback (most recent call last): 2022-03-01 07:39:18,220 Command udi_interface ERROR udi_interface:write: File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner 2022-03-01 07:39:18,226 Command udi_interface ERROR udi_interface:write: self.run() 2022-03-01 07:39:18,227 Command udi_interface ERROR udi_interface:write: File "/usr/local/lib/python3.8/threading.py", line 870, in run 2022-03-01 07:39:18,230 Command udi_interface ERROR udi_interface:write: self._target(*self._args, **self._kwargs) 2022-03-01 07:39:18,231 Command udi_interface ERROR udi_interface:write: File "/var/polyglot/.local/lib/python3.8/site-packages/udi_interface/interface.py", line 667, in _parseInput 2022-03-01 07:39:18,236 Command udi_interface ERROR udi_interface:write: self._handleInput(key, input[key], None) 2022-03-01 07:39:18,236 Command udi_interface ERROR udi_interface:write: File "/var/polyglot/.local/lib/python3.8/site-packages/udi_interface/interface.py", line 798, in _handleInput 2022-03-01 07:39:18,239 Command udi_interface ERROR udi_interface:write: self.nodes_internal[item['address']].query() 2022-03-01 07:39:18,239 Command udi_interface ERROR udi_interface:write: File "/var/polyglot/pg3/ns/00:0d:b9:53:2c:34_13/nodes/purpleair.py", line 86, in query 2022-03-01 07:39:18,241 Command udi_interface ERROR udi_interface:write: for node in self.nodes: 2022-03-01 07:39:18,242 Command udi_interface ERROR udi_interface:write: AttributeError 2022-03-01 07:39:18,243 Command udi_interface ERROR udi_interface:write: : 2022-03-01 07:39:18,243 Command udi_interface ERROR udi_interface:write: 'Controller' object has no attribute 'nodes'
Link to comment

You see a lot of these issues with the weather services for a couple of reasons.  

  1. The weather services tend to have a lot of data so I tried to minimize how much was sent to the ISY and most will only update when there are actual changes.  Because the i994 ISY has limited network resources, this is probably less of an issue with IoP, but since the node servers still support both, it's not going to change right now.
  2. The weather services tend to limit the number of API request so I don't enable interactive query since overuse of that could cause the limit to kick in and you'd end up getting no data until the limit was reset.
  3. In some cases, the weather services just aren't sending that particular data for your specific location.  In that case, the node server will send the default data when it starts and it will never send it again until it is restarted because it never gets updates from the service.
  4. Or like #3, the data is sent very infrequently. NOAA alerts for example are only sent when there is an alert published.
  5. Lastly, most of the weather service node server were written for a early, pre-alpha version of PG3 and do need updates to work better with the current version of PG3.  That's on the list but low priority at this time.

Also, since this is only an issue if the ISY is rebooted, the work around at this time would be to restart the node servers after the ISY is rebooted.  If you are having to reboot the ISY frequently for some reason, that is a bigger problem and should be addressed.

 

Link to comment
On 2/28/2022 at 12:24 PM, Michel Kohanim said:

This probably means that IoP starts much later than PG2 and PG3, and by that time, PG has given up till the next poll. We'll definitely take a look.

With kind regards,
Michel

With latest version today this seems to be resolved now after a full polisy reboot.

Restarting IoP separately still causes PG2 to require a restart while PG3 updates nodes over time (but a query also works to refresh them all). I am not too concerned about PG2 because I am using only one nodeserver on it which should be updated to PG3 soon.

 

 

 

Link to comment

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...