Occasional problems writing changes to KPL

[walkman9999]

Hi,

I'm having sporadic problems with an existing/stable install. Two of my KPL devices are giving "failed to write" errors about 50% of the time. Subsequent attempts will often succeed (although sometimes it take a few). Other comms in network are fine. I've attached two logs, one with success and one with failure. Can anyone give insight?

Edit: Should note: the KPLs work as expected as responders.

TY

NO-WORK-ISY-Events-Log.v5.3.4__Wed 2024.05.22 08.43.50.txt WORKS-ISY-Events-Log.v5.3.4__Wed 2024.05.22 08.44.42.txt

Edited May 22 by walkman9999

[IndyMike]

@walkman9999, I added some comments to your event logs.  In general, your communication looks fairly good (i.e. good hops remaining).

Sequence of events:

1. The ISY starts each write cycle by requesting a link table record from the device. 
2. In both cases (successful and unsuccessful writes) the device 2A.0B.38 acknowledges the request.  Both times the device exhibits excellent communication.
3. For some unknown reason, the device does not respond with the requested link record in the failed attempt.
4. The ISY retries 2 the link record read additional times, and the device does not supply the link record.

Questions:

1. What exactly are you attempting to write to the device?
2.  Can you perform a link table read/compare and post the results?

This is probably a case where you need to factory reset the KPL and restore it, but I'd like to see the link table comparison before we get ahead of ourselves.

 

 

[walkman9999]

Thank you for the insight. This problem occurred when changing attributes to a scene that includes the KPL, but all comms to this KPL seem inconsistent (e.g. changing ramp rate also resulted in a comm failure, when I manually retried to write to device it was successful)

Reading link table from the device was also problematic. First attempt retrieved 4 links and then failed. The second attempt (which is attached, along with comm log) failed entirely.

I do not mind going thru process of factory reset/restore. Let me know if more testing would help before I do.

Thanks again for taking a look.

LINK-TABLE-READ - ISY-Events-Log.v5.3.4__Thu 2024.05.23 07.24.03.txt

[walkman9999]

I am noticing that two other devices are having same problem (if I'm interpreting the logs correctly). Started yesterday with one of them (another KPL), and now today with a on/off switch (2477s). Logs are attached. 

 

The devices do write eventually, so far.

 

Editing to add. During this fail to write event, controlling the device seems to work fine. I've attached second log showing comms during on/off. And, I can read device links tables from devices not affected without problems.

 

ON-OFF-SWITCH-FAIL-ISY-Events-Log.v5.3.4__Thu 2024.05.23 08.14.47.txt

ON-OFF-SUCCESS-ISY-Events-Log.v5.3.4__Thu 2024.05.23 08.23.31.txt

Edited May 23 by walkman9999
adding info

[IndyMike]

2 hours ago, walkman9999 said:

Thank you for the insight. This problem occurred when changing attributes to a scene that includes the KPL, but all comms to this KPL seem inconsistent (e.g. changing ramp rate also resulted in a comm failure, when I manually retried to write to device it was successful)

Reading link table from the device was also problematic. First attempt retrieved 4 links and then failed. The second attempt (which is attached, along with comm log) failed entirely.

I do not mind going thru process of factory reset/restore. Let me know if more testing would help before I do.

Thanks again for taking a look.

LINK-TABLE-READ - ISY-Events-Log.v5.3.4__Thu 2024.05.23 07.24.03.txt 1.38 kB · 3 downloads

I was afraid of that. Given that you cannot reliably read a link table, please hold off on doing a factory reset on this device. I am not sure that you would be able to recover it. 

I Am away from home at the moment. I will review your files when I get back in the evening 

 

[IndyMike]

@walkman9999, your logs appear to show intermittent communication problems.  That's normally a noise source cycling on/off or varying it's frequency.

Do you have any new variable frequency devices (variable speed blower or condenser)?  I'm focusing on these items because I have a similar problem.  When my nice new variable speed condenser turns on and hits a certain frequency it shuts down communication to 3 of my devices in a very similar fashion.  What makes it tough to troubleshoot is the fact that it doesn't happen every time the device runs.  As loading changes, so does the frequency of the motor drive.  The problem only occurs at specific loads/frequencies.

Post back if you have any new items like this.

[walkman9999]

The electrical environment hasn't changed in any macro way, there is a chance that a new device has been plugged in outside of my control. 

Any suggestions for testing this theory? An unfriendly/noisy electrical environment seems difficult to fix. I have been considering purchasing a new KPL (if it doesn't help, having a spare seems like a good idea). Will post results of the swap once I receive it. 

Thanks for your continued input.

 

 

[IndyMike]

6 hours ago, walkman9999 said:

The electrical environment hasn't changed in any macro way, there is a chance that a new device has been plugged in outside of my control. 

Any suggestions for testing this theory? An unfriendly/noisy electrical environment seems difficult to fix. I have been considering purchasing a new KPL (if it doesn't help, having a spare seems like a good idea). Will post results of the swap once I receive it. 

Thanks for your continued input.

 

 

Having a spare KPL is never a bad idea.  I had considered that you might have a KPL with a failing power supply.  It's doubtful that you have have 4 devices failing in a similar manner at the same time - unless you had a recent power event.

I had also considered a failing PLM.  The fact that you can communicate with your other devices and the communication problems are sporadic make this unlikely as well.

You are correct that tracking does a noisy device can be difficult.  This is particularly true when it's an intermittent source.  I would start by looking at the devices that are being affected:

1. Are the devices all on the same circuit - or widely spaced.  If they are on the same circuit, begin disabling other devices on the circuit to eliminate possible offenders.
2. Are the devices all on the same phase of your electrical system.  Running a 4-tap beacon test can help you determine this.  Noise will not normally cross an electrical phase - unless the offender is a 220V device like my A/C condenser.  If you find that your devices are on the opposite phase from the PLM you can move the PLM, Improve the RF phase coupling, or add a passive coupler.
3. I'm assuming that your issues are due to intermittent noise on the powerline.  If you have plug-in LampLincs or range extenders, you could try to improve RF communication by moving one nearer to your problems device(s).
4. Are your problem devices all dual band (I'm guessing they are since they appear to be I2 capable).  If not, absolutely move a RF device (range extender or Lamp Link) nearer to the device.  Alternatively, upgrade the device.
5. Look at your PLM placement - are there noise generators/signal absorbers near by (UPS, PC, chargers).  An easy test would be to move your PLM to a different circuit.
6. Ask your self what type of demand devices you have that operate automatically (HVAC, Vent Fans, Well Pumps, Sump pumps, outdoor photocells).  Look for correlation between the operation of these devices and your issues.

There really aren't any great diagnostic tools for tracking down noise devices.  Most troubleshooting uses the process of elimination (unplugging devices, turning off breakers) and observation (correlating problems with other device operation).

 

 

[walkman9999]

This is all great advice and there is a lot I can work with here. KPL is next (should be here in 5 days), and I'll put some of the easier things on this list into my "best practices" list.

The workaround of writing until it "sticks" should make this operationally acceptable, especially since the switch and KPL commands seems to work fine for now and I don't change things often.

Will report back on new KPL for closure. Really appreciate the knowledge transfer and targeted troubleshooting.

 

[walkman9999]

Tentative results seem to indicate that the the new KPL has solved the problem. There is still the matter of the other KPL that had similar symptoms, but that one is behaving lately. Acknowledging that there are some logical leaps that have to be taken to blame the KPL, but for now that is the conclusion.

 

Thanks again @IndyMike for assist.

[IndyMike]

@walkman9999, happy to hear that the replacement appears to have solved the problem.  Not sure that I in any way helped with that...

If it was in fact the KPL power supply that was failing, you may have uncovered a troubleshooting tool - link table reads.

Link table reads/writes are performed on newer I2 devices using the "extended" communication mode.  Normal on/off commands are performed using I1 "standard or direct" communication.  Under normal conditions, I2 communications are more reliable since they transfer more information faster and with far fewer handshakes (less opportunities for failure).

Your devices were exhibiting the opposite - I1 (direct) communication was more reliable than I2 (extended).  Your on/off commands functioned OK while link table reads/writes failed miserably.

Since I2 communication transfers 2.5x the number of data bytes, it also requires sustained power on the part of the KPL to transmit.  It's possible that the power supply caps in your KPL are failing (similar to the PLM power supply failures) resulting in a lack of capacity to maintain the I2 communication.