bpwwer

What do you get if you load that URL listed in the log snippet above into your browser? Can you post or PM me the results? Also, it would help to see the full log, including the node server startup so restart the node server, download the log and PM it to me.

I gave your profile files a pretty good going through and I missed it too.

There is probably an error in one of the profile files that is preventing the AC from fully parsing the node. Looking at your profile files, I don't see anything wrong. The best way to double check is to start the admin console with the java console active, any parse errors will be reported there.

The "Pool thermostat" even though it's a child node will show up as a separate node in the program dropdown. The dropdown doesn't support displaying the hierarchy, it just list the nodes; parent or child.

go into the log page and download the log package. PM me with the log package .zip file

What version of PG3 are you running? Is there anything in the log that would indicate why it's not able to get data from the tempest?

The duplicated lines is just because PG3 has multiple streams open to the log file, reloading the page in the browser should close those and resolve that. Try enabling debug log level to see if provides more information on why the queries are failing.

Usually that means you've exceeded the number of queries allowed for you plan. It seems like some of these services have been making that limit lower over time. It will usually reset on a daily basis so you might want to reduce how often you poll and wait a day.

I need to try and reproduce it.

Thanks for the details. That really does help. The node servers are verified every 5 minutes so that's probably not related, just that it is what you'll see in the log when PG3 is mostly idle. But the reconnecting to MQTT broker is related. That's saying that the network connection between the MQTT broker and PG3 dropped for some reason and it had to reconnect. Most likely , that also means that the network connection between each node server and the MQTT broker also dropped and it's each node server's connection dropping and reconnecting that is causing the notifications. Since the MQTT broker is a separate system service communicating with node servers and PG3 over the network, can you think of anything on your network that would be causing network disruptions on a 8 hour frequency?

Realize that currently, the PG3 backup/restore doesn't handle the fact that the IoX may also be changing so creating a backup of eisy and trying to restore to a Polisy (or another eisy) doesn't fully work. The backup/restore processes were written with the assumption that PG3 was pointing to a ISY994 and that if you replaced the Polisy, you'd still want to new Polisy to point to the same ISY994 (this was all done before IoP/IoX existed and hasn't been updated since). Fixing this is on the todo list.

Doing a restore will simply try to re-install all node servers so I wouldn't expect it to behave any different. Since it will re-install all node servers it's possible that it could cause this same issue with other node servers. So I'd say more risk of of making things worse instead of better.

If it is only one of many node servers then it is likely something happening with the node server. Node server will attempt to re-connect to PG3 automatically if the connection drops so you may not notice anything wrong, but this will now trigger notifications to UDMobile. In this case, you should see a "failed" notification followed by a 'connected" notification. If you are seeing notifications for all node servers at the same time, it is likely PG3 that has crashed. PG3 will automatically restart so again, you may not be seeing any other indication of a problem but when PG3 crashes, it will attempt to stop all node servers before restarting and both the stopping and restarting of the node servers will trigger the notifications. In this case you should see "disconnected" notifications followed by "connected" notifications. In either case, this is an indication that something is wrong. Node servers shouldn't randomly disconnect and PG3 shouldn't randomly crash. If you just don't care because things seem to be working fine, you can disable the notifications by going to the System -> Notifications page and disable PG3 notifications.

Yes, it is strange. I installed it here fine and then re-installed it and still fine. I've had another report of the same problem, different node server.

Yes, the missing file would be the problem. Try installing (re-install) it again.

@mbking I realized that hardware replacement would cause issues while answering your post so I added that to the issue list. If all node servers are disconnecting/reconnecting at once, that most likely means that PG3 restarted. There's another thread (I think the support thread) where this was reported by someone else. In their case it looks like the network connection between PG3 and the broker failed (reason unknown) which is a fatal error in PG3.

I believe one of the recent system updates does remove PG2 from the system. I don't know if @Michel Kohanim announced that or not. I had plans to include RIO support in my Russound PG3 node server but that hasn't made it on to my active tasks yet.

The PG3 (and PG3x) backup/restore process was never meant to migrate. Hence the need for a specific migrate from PG3 process. The backup/restore process assumes that the IoX (or really the ISY994) wouldn't change between a backup and restore as it was all developed when only ISY994 existed. I think the only way forward is to re-install each node server and re-configure them. You can re-install to the same slot over the unmanaged entries.

Yes, I've been monitoring it. There is a proposed fix, but the code base hasn't been changed since 2019 so I'm not real hopeful anyone will actually do something with it. I may have to fork the code and make the fix myself. That's something I can look at doing in a couple of weeks.

Like I said, there are only 2 cases where PG3 will tell a node server to shutdown. 1. Someone clicks the stop or restart buttons in the UI 2. PG3 shuts down. In your case it looks like PG3 shutdown: 5/26/2023, 13:59:57 [pg3] error: uncaughtException: read ECONNRESET Error: read ECONNRESET at TCP.onStreamRead (node:internal/stream_base_commons:217:20) 5/26/2023, 13:59:57 [pg3] error: uncaught ECONNRESET exception: Error: read ECONNRESET at TCP.onStreamRead (node:internal/stream_base_commons:217:20) 5/26/2023, 13:59:57 [pg3] info: Caught SIGTERM/SIGINT Shutting down. 5/26/2023, 13:59:57 [pg3] info: Stopping running NodeServers... That error indicates that the MQTT broker unexpectedly closed the network connection and since that connection is not supposed to close unless PG3 exits normally, PG3 simply aborts and restarts. As to why, I have no idea. The broker logs may have some information, but I've not found them to be very helpful in the past. Could also be something external disrupted the network (router reboot, etc.).

From the log it looks like your Sonos is refusing the connection attempt from the node server. After a few tries, the node server gives up and aborts. So it looks more like a Sonos problem than a node server problem.

It's not real clear what you're actually running. PG2, which runs on a RPi and on a UDI Polisy is no longer supported. However, it should continue to run. Most of the PG2 node server are also no longer supported. There is a WeatherFlow node server for PG2, but it may or may not work. Again, if the node servers continue to work, you're fine for now. PGC was discontinued and is no longer active, this includes all the node servers for it of which WeatherFlowPGC is one. PG3 and PG3x are the latest versions of Polyglot. These will only run on a Polisy or eisy controller from UDI. So you can't install it on a Raspberry Pi. PG2 node servers are not compatible with PG3(x) and many are no longer free. The transition from PG2 to PG3(x) has been happening over the past 2+ years.

@dbwarner5 stopping and starting the node server are supposed to generate notifications. On stop, it disconnects and on start it connects. Those are normal and correct. Stopping and starting node servers should not be happening often. In fact, the only way the node server can be stopped if if you stop it or PG3 shuts down. What I'm concerned about are "failed" notifications. Those mean the node server stopped communicating with PG3 which is something that should not ever happen and if it does, it's probably a bug somewhere.

Please verify in the node server logs that the node servers are actually disconnecting and reconnecting. You'll have to do this within the same day the notification was sent. In my case, the node server was disconnecting and reconnecting so the notification system was working as intended. Also, the node server was working as intended. When the connection dropped, it reconnected and the node server continued working without issue. The question is what's causing the connection to drop in the first place? In my case it appears to be caused by network packet corruption or possibly a bug in the low level networking code. In either case, not something I can fix.

You should update to the latest version of PG3 as this issue has been fixed. The (Re)Install button isn't really correct. It should be labeled "Update" and you can only update if you already have it installed. Instead, go to the Node Server Store, select the node node server from the list and then click the "Install" button to install.