EDIT: NO JOY, problem recurred within 24 hours. See here, and down thread ...
https://forum.universal-devices.com/topic/46764-multiple-devices-become-unresponsive-not-solved/

--

I have recovered from this, I think, but can't be certain there still aren't some issues. I am wondering if there is something more I should do, like certain diagnostic steps, or even potentially replace the PLM.

In some 15 yrs of ops, have never seen quite this kind of failure.

The System

System is large, 100+ devices, mostly insteon, a bit of zwave (700 series dongle), polisy running "LTS" 5.4.4. Zwave was added during the time of insteon troubles. Stuck at 5.4.4 as the polisy OS won't support further upgrades, and the bit of Zwave now complicates a move (reflash/restore etc.), so been sitting on it for a bit.

The Problem

One morning I noticed that ALL of my wireless insteon devices stopped working (ISY would not see any transmits), that includes the 2842 motions, the newer 2844 motions, and the mini two & four key 2432 remotes.

But, in addition a number of the 2334 eight/six button units would also not be seen by the ISY. Curiously, scenes that toggled the state of the buttons on these, still functioned. (so ISY saw no TX, but units accepted RX)

Zwave devices not affected.

In the UI, no device showed as not responding, or any indication of offline status, except that the wireless devices' pages showed blank fields, that is no ON or OFF status, etc., and not able to QUERY or change settings.

The was no "nine blink" indication on the motion devices (indicating network connection loss), nor the "three blink" showing low battery.

What I Did

Scratched my head for while, checked the diagnostic screens, nothing obvious to me.

Then on a lark, starting with the simplest thing, I put one of the wireless devices in link mode, and did a simple RESTORE DEVICE command. I saw the usual activity dialogs, and then amazingly, the device started working normally!

I then tried this with one of the 2334 eight button panels, that restored its function as well.

I went through each of the devices (that I know faulted), and did the same. They all started working.

The Question

How did this happen? Do I need to do anything more?

Were all of the wireless devices, or matching tables in the PLM, somehow selectively memory corrupted? But, what about the eight button panels too?

Is the PLM starting to fail. Certainly I've seen random devices fall off for control when that happened in the past, but never such a logically grouped set of devices, nor could I restore function as easily. With the device restores the system seems to be back up and running. Pretty sure the PLM is newer version, with the updated capacitors.

Thoughts??

Orest

Edited 38 minutes ago38 min by oskrypuch
removed link to merged post

Congrats, you win the odd-one-of-the-week award!

Initially I was thinking rf noise, but reading through it all, it would appear that the PLM memory got partially corrupted. If, as you seem to be saying, wireless devices were still able to trigger actions in other devices (as part of scenes) but nothing was seen at the ISY, it seems that links would have somehow been lost in the PLM. If I had suspected this, I would have started with a restore of the PLM. Of course, hindsight is always easier, and I might be totally off here.

Yes, I was thinking the same, an odd one!

I am suspecting the PLM, but so odd to have such a specific grouping of devices all fail at the same time. Maybe some sort of "logic wave" that went through the system, and it affected only certain kinds of devices. A stretch I know.

I was going to try the PLM refresh, but it takes a bit of time, so thought why not -- try the simple device restore first. I would expect that restores the device's entries in the PLM tables, besides the device itself.

Maybe I should still do a PLM refresh, to catch things I haven't noticed yet.

Orest

Edited Monday at 06:53 PM4 days by oskrypuch

Spoke too soon!

Same failure, again.

So perhaps a PLM refresh, and if that doesn't work (or maybe just do it as I have spares), a PLM replacement/refresh.

Orest

Edited Monday at 07:36 PM4 days by oskrypuch

Oh dear, last 2413S I have (v2.5) I have it marked with tape saying "bad?".

And they don't sell serial PLMs any more.

Need to go USB modem, so that means I remove the current serial cable, and just plug the USB cable in directly?

Orest

Edited Monday at 07:58 PM4 days by oskrypuch

A Polisy has two USB ports, so you should be able to just plug it right in. Yes, the 2413U comes with a standard USB cable.

Yes, I just checked. I knew that there was at least one, as I have the old 700 series ZWave dongle plugged in. I feared there wasn't a second.

Orest

Now to find the docs, so I get the sequence exactly right. Can't find "replace PLM" or similar, in the wiki. I know it is there somewhere ...

Anyone?

Orest

The first post in this thread details the procedure. The important thing is to power off the controller, as explained:

EDIT: Replacing the PLM did NOT work

--

OK, found the original PDF as well (clipped below). I understand it is critical to have Polisy discover the new PLM, and not use the old PLM address, so the reboot.

I might just give a refresh of the PLM (without replacing it) a try as well. In that case, I start at the dashed line I added in below?

Orest

To replace a PLM:

Unplug ISY from the power outlet

Unplug the PLM from EISY/POLISY and power outlet

Connect the EISY/POLISY to the new PLM

Plug the new PLM into a power outlet

Plug ISY into a power outlet

Go to Admin Console and wait for system initialization to complete

. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(to just refresh a current PLM, start here)

Click on the Battery icon at the top. This will prevent EISY/POLISY from trying to update programming on your RF devices which are probably in sleep mode

Click on the Restore Modem (PLM) menu option

In case EISY/POLISY cannot communicate during the process, you will have a series of nodes with green 1011 icons in the device tree. For each one of those nodes:

If an RF device, ensure that the device is in Programming mode

Right mouse click | Write pending updates ..

Edited yesterday at 04:01 PM1 day by oskrypuch

Sounds about right.

12 hours ago, oskrypuch said:

The Problem

One morning I noticed that ALL of my wireless insteon devices stopped working (ISY would not see any transmits), that includes the 2842 motions, the newer 2844 motions, and the mini two & four key 2432 remotes.

I had the same thing several days ago. Very small Insteon-only system on an ISY. My wireless devices were two open/close sensors and two leak detectors. The two open/close sensors drive two variables that trigger some main functions. No response from wireless devices in the admin panel. No variables changed or programs triggered.

I restarted the ISY. No help. Repowered the PLM and the ISY. No help.

I begrudgingly did a PLM restore. The ISY prompted a relinking of all wireless devices. I tried to relink one device, but that leak detector started responding before I could finish. The other three wireless devices started responding, too.

The system has been working fine ever since (3-4 days.) I suspected the PLM may be going bad. Now, I dunno what to think. Weird

@auger66 Well, I think you are in the "interlude", at the moment. 😉

I would suggest, if you have a recurrence, buy a new PLM and carefully do the full the replacement procedure. (as per above)

OTOH, if it doesn't recur soon, if you have an older PLM (pre v2.4 or so), just get a new PLM anyway, and have it in inventory for the future.

I have ordered a new 2413U, should be here in a couple of days, and will report back. For now my smart house, is semi-dumb!

It seems that the fault state, affects any device where the system expects to hear from it with a Tx. Obviously that includes wireless motions, remotes and water detectors, but that also includes the 6/8 button panels and similar. And the fact that a device restore (temporarily) corrected the issue, to me strongly suggests it is a hardware rather than logic/software issue.

All points to the PLM, the usual weak link.

Orest

Edited Tuesday at 06:34 PM3 days by oskrypuch

A new 2413U probably would be best. Even if stored for a backup.

The 2413S serial PLM used by older ISY controllers has been discontinued by Insteon so older users may need to find a new way to replace the PLM or a rebuild.

A Polisy will still accept the DB9 serial adapter, from a 2413S, that is actually what I have currently.

There was a serial/USB adapter for the Eisy as well, but not sure if it is available any more, either.

But for sure, best to get a 2413U, rather than trying to ebay out a 2413S combo.

Orest

Edited Tuesday at 06:39 PM3 days by oskrypuch

"Smart" house is coming back alive. Still have to crawl around and get the wireless water sensors, but then it will be done.

Just needed to replace the PLM. Bought a new 2413U, got it connected to the Polisy, and initialized as per the procedure.

EDIT: Not great, identical problem recurred after 24 hours with the brand new PLM.

I am back to square ONE.

Edited yesterday at 03:56 PM1 day by oskrypuch

So, in a nutshell, a large, mature Polisy install, with a few ZWave devices controlled by the old 700 series dongle.

Just recently, noticed a fault, 100% consistent, and I've now defined it definitively:

• Any transmissions from peripheral insteon devices do not reach the Polisy, and are not acted on. That includes both wireless and wired devices, so motions, remotes, but also panel buttons

• Transmissions between insteon devices (shared scenes etc.) work fine

• Control of insteon devices, that is insteon device reception of signals from Polisy, works fine

• Activating programs or scenes from either the admin UI or the android app work fine

• ZWave devices are not affected, work 100%

A DEVICE RESTORE of a given insteon device will result in restoration of function, but only briefly, perhaps 12 to 24 hours.

Thought the PLM was going bad, replaced it with a new 2413U. Fully restored all functions, then bizarrely the next day back to the identical fault condition.

I rebooted the system a couple of more times, for good measure, that actually made no change.

Network noise seems unlikely, as it is only one way communications that is an issue, and there is no delay in that, it is "crisp". Further I used a remote button device within a couple of feet from the PLM, so the hop would have been wireless entirely, and it did not work.

I'm not an expert on the event viewer, but from what I could see the system event viewer did not show any inbound activity when triggered, but normal outbound, reflecting what I am seeing with the system.

It strikes me as unusual that it takes a while for the fault to cover, once temporarily mitigated, some sort of buffer over-run or something? Yet a reboot didn't help.

@Guy Lavoie -- you were right to call this one an odd one, it may be the odd one of the year!

I was thinking of refreshing the IOX install, but would have to find an old copy of 5.4.4, can't easily move beyond that as the OS in the Polisy is older. The early ZWave controller has been a bit of an anchor in that regard, as those definitions would have to be all redone, if I update.

Is the Polisy dying, bad memory, or some component thereof?

Running out of things that could be at fault.

Disheartening, on top of losing the function of the smart home niceties.

Orest

Edited yesterday at 05:14 PM1 day by oskrypuch

One possibility is that the old PLM was dying over time.. this has happened to me. The way the polisy works is that it copies link info from the PLM to its master link table when it sees changes. As the PLM functions fail, the links get scrambled or polluted, and the polisy copies that in.

When you set up the new PLM and used restore modem, possibly bad link data was copied from iox into the new PLM.

Since your Insteon devices are all working good stand alone, it's plausible that a restore device could force the new PLM back to reality. Start with single switch that only has a few links (eg not a keypad or wireless devices) and right click in iox and pick "Restore device"

If that works, slowly go uphill to switches with more scenes or keypads, stopping to test after each.

However it's possible that some devices will not respond to this. If that's the case, start by cleaning out and redefining the scene. If that's doesn't work its possible that the device needs to be deleted and re-added

Edited yesterday at 04:49 PM1 day by paulbates

14 minutes ago, paulbates said:

One possibility is that the old PLM was dying over time.. this has happened to me. The way the polisy works is that it copies link info from the PLM to its master link table when it sees changes. ....

Thanks! An interesting idea, and I'm game for anything.

So, Polisy has two sources of link data? That is, if it copies (possibly faulted) data from the PLM to its table, where would it get fresh "correct" data to restore with?

The time lag, from working (when restored), to not is a little hard to explain.

With the old modem in place, doing a RESTORE DEVICE, (always) worked but only for a day or so.

But, I actually have done exactly that (RESTORE DEVICE) on several devices, with the new PLM in place, more just to restore their function! But, I was going to watch them to see when/if they stopped functioning.

Will report back.

Orest

Edited yesterday at 05:14 PM1 day by oskrypuch

Weird that it apparently worked ok for a day.

On a lark, I'd try plugging the PLM into a different electrical circuit. Use a long extension cord to have it connected to an outlet elsewhere that you know is working ok with your other devices. I'm just wondering if there is something on your existing circuit that could be killing the signals.

3 minutes ago, Guy Lavoie said:

On a lark, I'd try plugging the PLM into a different electrical circuit. Use a long extension cord ...

Yes, worth a shot.

The current outlet is an isolated circuit, straight to the panel, for small current electronics only. And it would be odd to hit only inbound PLM traffic, but we are into the Twilight Zone on this one.

Orest

Edited yesterday at 05:19 PM1 day by oskrypuch

@paulbates It may well be that the PLM replacement function uses a different data table from the device restore, which restores both the device and PLM.

Orest

Edited yesterday at 05:25 PM1 day by oskrypuch

I'd try that simple, isolated example. Based on it working, or not, you can go from there.

When you developed a "similar" issue, with a gradual loss of your PLM, was that the ultimate solution for you?

Orest

Poking around the link tables, here is a capture of the device and ISY tables for a given device, when it is not working correctly ...

And here it is, AFTER a device RESTORE (as per above) of that device (and return to functioning) ...

They match now, and it is the device that was changed, the last three entries were deleted, and a high water mark was added.

I don't know exactly what that means, but probably someone here can decode that.

Now, to see if the disparity in the tables (and loss of function) recurs.

We might be getting somewhere.

Orest

Edited yesterday at 07:18 PM1 day by oskrypuch