-
Why did multiple devices become unresponsive?
Ok, good to hear, and I now have the data dump so can review in future. Thanks for checking it out. Looks like I am back in business! Orest
-
Why did multiple devices become unresponsive?
Well, I have the UI's XML dump from the 22nd, 23rd and 25th. The latter two, the 23rd & 25th are identical, the initial one on the 22nd differs slightly, as I was commenting. I've attached the two that differ. Orest PLM Links Table.v5.4.4__Mon 2026.06.22 11.57.31 AM.xml PLM Links Table.v5.4.4__Tue 2026.06.23 01.40.12 PM.xml
-
Why did multiple devices become unresponsive?
@IndyMike Well, it has been a day or two, and all is running normally. But, I did notice a difference in the PLM table. Same number of items, and the "id" three bytes, and the "data" two bytes are constant, but in a half dozen (of the 293 lines), the FL & GR values, which is the first two bytes, did change. I am thinking that this is normal? Orest
-
Why did multiple devices become unresponsive?
Python is easy to pickup, quite powerful, and available on most every platform. Less popular these days, but still very handy with a rich library of code. Orest
-
Why did multiple devices become unresponsive?
That is some impressive sleuthing. The log files are all txt, so easy to manipulate. Did you write some code in C or python or something? I have the original Insteon design white paper (somewhere), and was going to dig into it years ago, but got distracted by something else. Orest
-
Why did multiple devices become unresponsive?
Guy beat me to it (looking), but just wondering, have you tied the 1/2 sec (vice 1 sec) delay to errors in data Tx? Orest
-
Why did multiple devices become unresponsive?
The few PLM entries I added did not change, or vanish, overnight, and function on these devices is preserved, so ... Completed a full PLM restore (293 lines now), and now doing a full device restore. Looking good, so far. Orest
-
Why did multiple devices become unresponsive?
And, I'll keep the device event viewer/log rolling for now. Orest
-
Why did multiple devices become unresponsive?
Well then, thanks to @kclenden too! And, thanks to @Guy Lavoie and @paulbates who helped get the ball rolling. The time frame of that errant code insertion, does match up with the observed faulting issue, so good circumstantial evidence. I'm going to sit with just the few devices I have restored for now, and confirm that they stay fully working -- a day should do it. There are only 32 PLM table entries right now, so easy to download/compare. If no further resets/issues, will do a full PLM restore, and as you suggest, follow that up with a DEVICE RESTORE on each device, for good measure! Thanks again. And, will report back. Orest
-
Why did multiple devices become unresponsive?
@IndyMike I am so grateful for your time and expertise on this! I think we are well on the way to solving this issue. Having the PLM (errantly) reset from time to time, would explain everything we are seeing. Do you feel there is a good chance that this comm overload triggered the errant reception and reset? The every thirty seconds thermostat polling was not intentional. There was a (recent) bit of code I added, and it looks like it got into a loop from time to time, I've squashed that, and checking the device comm log, the severe polling of the stats is stopped. I also do monitor stats that are calling for heat and cool to "count" the time of operation, but this is not new, and this executes once a minute, only if a stat is calling. For good measure I've disabled this for now. If I reenable it, will move it out to five minute intervals, that is accurate enough. Runaway code is obviously a bad thing in general, to say nothing of triggering an ALL-ON issue! As a test, I have DEVICE RESTORED some usually affected devices, and saved out the PLM TABLE for comparison/reference. And now I know precisely what command to look for in the log, if I see that fault again. If that sorts it out, I'm just a PLM restore to be back to normal! Will report back, and happy to hear any other suggestions you may have. Orest
-
Why did multiple devices become unresponsive?
... and that single DEVICE RESTORE, restored the function of that one device. So, I am thinking, that the 70 or so entries of the PLM table (copy uploaded), really just represents the PLM entries from the few devices I had just DEVICE RESTOREd. Given the number of devices I have, I might expect to properly see many hundreds of PLM entries. But, what is causing the mass clearing of the PLM table!? Is a failing Polisy, the problem? Orest P.S. And, there are NO [ 02 6F ] entries in the log.
-
Why did multiple devices become unresponsive?
@Guy Lavoie Correct, to be absolutely clear, all scheduled commands from the Polisy to devices are working ok. -- AND, the fault trigger just occurred, all devices can no longer transmit, including the ones that I just recently DEVICE RESTORED. AND, looked at the PLM table, it was BLANK! Hit START a number of times, no change. I then DEVICE RESTOREd one device, some PLM links now show up in the PLM table, eleven of them. Well if one device gives eleven links, clearly the PLM table is way under populated, even when I thought it was "full". DEVICE RESTORE a second device, now there are 29 entries in the PLM. Something is clearly "rotten" in the PLM table (or access to the PLM table by the Polisy) when the fault occurs, and that is shutting down devices communicating!! @IndyMike Attached is the original 70 entry PLM table, which is likely "light", and the event log that covers today, including the period when the fault occurred, which appears to be an emptying of the PLM table! PLM Links Table.v5.4.4__Sat 2026.06.20 11.50.29 AM.xmlISY-Events-Log.v5.4.4__Sat 2026.06.20 02.05.23 PM.txt
-
Why did multiple devices become unresponsive?
For a temporal overview of this ... 1) State: normal, all working -> some kind of trigger occurs (seems to happen every day or so, now) 2) State: all devices can no longer successfully transmit packets to the PLM/Polisy, no other functions affected -> individual RESTORE DEVICE command on a given device 3) State: that one given device is restored to normal function, all other devices still remain faulted -> some kind of trigger occurs, again 4) State: ALL devices once again can no longer transmit to the PLM/Polisy. Orest
-
Why did multiple devices become unresponsive?
@IndyMike Do not apologize for a loooong post, I love it! The PLM table shows 70 entries (table is now saved out), presume that means it sees 70 insteon devices. That is about right, there are some more ZWave devices as well which obviously are not reflected there. Cable - this fault has survived through two PLMs, and also through two different cables, one the DB9 serial cable, and now the USB cable, so that eliminates cable, connectors and plugs as an issue. A second PLM restore (or even just a serial device restore on each device) is on the plate here for sure. I obviously did one PLM restore, when I installed the new 2413U PLM. For now, I once again device restored one 2334 keypad (yesterday) to monitor, it is still working some 14 hrs later, will see if a full 24 hours results in the fault. Also, today I device restored two more 2334's, four 2477S switches and one 2844 motion. Why those? ... The 2334s partly display scene status (that works fine), but some of the buttons also trigger programs so send a packet to the PLM/Polisy. The 2477S manual switch status is used as a logical trigger for some programs, so that requires status transmission to the PLM/Polisy. And, the motion of course sends packets to the PLM/Polisy for action. Those were chosen as a sampling, as anything that requires a packet from a device, to tx to the PLM and the Polisy, fails when the fault occurs. Once device restored, they resumed full normal functioning, for now. As noted, I have now saved out the PLM table, 70 entries, but curiously no NULL end of list item, should there be one? Perhaps I don't have the full listing captured, but I did try a few times. I am running the event viewer in mode 3, there will be a lot of data there! The one huge advantage with this otherwise annoying issue, is that it is100% consistent, so potentially can be debugged. 100% consistent, when the fault state occurs (no idea what triggers this) all devices in the system that need to transmit a packet to the PLM, and then Polisy, for action don't, or the packet is lost/dropped in transmission. I was watching the event viewer, and pushing buttons of devices when faulted, show no device activity, which makes sense. Because all devices that use this mode (Tx), lose this ability all at once, as has been suggested, it surely has to be something up stream, the PLM or Polisy. Also 100% consistent, once all devices are faulted, a DEVICE RESTORE on a given device will restore it to normal function (for a while, until the next "trigger"), and it will now get its transmission packet received by the Polisy, which then carries out its action. Nothing else, that I have been able to discern is affected. The system otherwise is working. Programs are running, executing commands, scenes external to the Polisy continue to work, as is direct control of lights from manual operation of switches. The one obvious exception to programs running properly during the fault, are those that rely on a report of state of devices (switch positions, motions, etc.), the logic may fail as the state is not correctly updated at the PLM/Polisy. And using the UD mobile app to trigger programs works fine, as a surrogate for example, of a curtain open/close button push of a 2334 keypad. But, you would expect that. Will report again tomorrow, and post the logs. Orest
-
Why did multiple devices become unresponsive?
Although I thought earlier, that the change in the device link table was pertinent, presently, with no change to the link table the fault occurs. There were some stray and duplicated entries previously, but they were not significant to this issue. And now (with the device link table "fixed"), the device restore function doesn't actually end up changing anything in the device's link table, yet results in an immediate restoration of the function of the device. I understand in a global sense what the device restore function does, but perhaps there is some internal data table or cache fix up that happens as well. Sure I am grasping at straws, to try to explain (and mitigate) what I am seeing. I just now have "device restored" one 8 button panel, it now works, for example one of the buttons opens/closes the curtain, button pushes from this device are now seen by Polisy and the correct program is triggered with effect. With that device restore the device link table did not change, so it must have done or triggered something else to happen. This fix up effect is immediate, and so far, 100% consistent -- once the fault occurs. I will be monitoring it for function as well as any change in the device link table. If it continues acting as devices have in the last few days, by tomorrow it will be no longer working. I'll also look at giving the Polisy a 24hr power off "rest", to see what happens. Orest
oskrypuch
Members
-
Joined
-
Last visited