Illusion

Polisy. IoX version 6.0.0

1.4.3

If my UD Mobile app is using a remote connection, the home screen at the top will say "Disconnected, will reattempt connection" It will show an attempt counter eventually just landing at the disconnected message. But it appears to be connected. If I go into controller settings is reports as 'Online' with Synchronize working fine. Devices show correct status once I focus on them, but do not auto update which is the behavior I have come to expect when on a remote connection vs the auto status updates I see when local. It does not matter what device I use, and it does not matter wether that remote connection is cellular or wifi. Running v1.4.3

It took me 4 tries to reinstall and get it working again. But then no configuration work or anything. Poof all working again! Thanks for this thread!!

Upgrade of OS to 14.1 and IoX to 5.9.1 has corrected the issue. I guess we never get to know what was going on. I am going on the theory that it was something in IoX 5.8.4 as my update to that firmware coincides pretty closely with the start of the issue. Why no one else had the issue, and why we could not figure out what mechanism was the cause will remain a mystery.

Upgrade to wired Polisy went well. Only notes are I did not get the 4 beeps at the end. Did get beeps throughout. Took about 20m PG3x and system no longer show up at https://polisy.local:8443/WEB/sysconfig.txt) and https://polisy.local:3000 I have to use the IP address.

I am going to move away from system startup time research. I am going with @bpwwer view that this is an admin console bug. Reboot/restarts later in the afternoon were still 41s off, but the AM/PM differential went away. Based on the fact that the time displayed in the admin console, that ssh shows Polisy knows the correct time, and that the email I get from ISY on reboot is timestamped correctly in the message body by the ISY, I do not want to spend any more time on this. I did 6 hours of tests on this issue yesterday. I am not opposed to coming back to this, but I would like to focus my efforts on the fact that IoX does not restart if I am not connected to the UD Portal. I think the answer lies in that. Either that or figuring out what is different about stopping internet access for the IoX vs other forms of internet interruption. I think that could give us a great data point. Data point #4: and Variations on those tests #3: @Geddy Regarding your idea of changing networking cable and topology, I thought that would be a good thing to take as far as I could while testing the startup time anomaly extensively. So I strung ethernet cable together to run my Polisy on my neighbor's network. So different WAN, different LAN, different cable, different router. Same results. IoX restarts unless I break the connection to the UD Portal.

Okay, here is what I currently have investigating the time/NTP issue. @Guy Lavoie Thank you for that suggestion. Can confirm that Polisy has correct time as verified through your ssh suggestion. The time is not always so far off. I restored internet at 11.50.30am and system shows: I restored at 12.02.30p and system shows: I restored internet at 12.09.03p and system shows: I am not sure this strangeness has anything to do with my issue at all, but is fascinating. I was wrong about this only happening with the internet drop and restore. There is the same time discrepancy if I reboot the Polisy or 'Restart IoX' from the configuration pane. The same discrepancy shows up if I do a remove/restore power to the Polisy. I had missed this by not paying attention to the AM/PM or because all those test were done in the AM and being off by a few seconds during a reboot is impossible to detect because of system startup. It is off by a few seconds and always seems to show only AM. I will do more testing in the afternoon.

@BigMojo Oh wow! I had not even caught the AM/PM differential. Interesting

@Geddy Just tested with different network cable connected to a switch connected to a mesh node in another part of the house. Same results. @gregkinney 1. Network cable replaced. 2. Tested on completely different power supply. A battery, so no wall outlet or circuit breaker involved at all in this test. Full isolation from home power. Same results. New Data point! I was doing all these test by just pulling the ethernet cable out and plugging it back in since I had taken the effort to get to the back of my Polisy to change/reroute the network cable. Most of my tests are done by shutting off the WAN in the router just because it is the easiest. But what I realized on one of the restarts, the restart time did not seem right. Because it is a physical removal and restoration, I can know the exact second that the internet was restored. Internet was restored at 12.06.15p. IoX System status shows a startup time of 12.05.34!!! IoX is showing a startup time 41s BEFORE the ethernet cable was plugged in! I do not know if it is relevant, but I know computers like correct time. If I select the restart IoX button, or I hard reboot the Polisy, the startup time is correct. (The system time in the Admin Console is correct in all cases)

My Notification plugin will always notify all my devices that have UD Mobile with a message "Notification Node Server Standard Edition Startup." any time the Notification plugin starts up. I do not seem to be able to stop this. Other messages like "PG3 is starting" respond to the "Plugin Notifications (PG3)" group, and messages like "(Polisy) is now offline" respond to the System Notifications group.. But I do not seem to be able to stop the notification from this plug in. It goes to every device that has UD Mobile. Is there something I am missing?

That is a good one! I started to test this as a factor but stopped when I realized that the test where I unplug the ethernet cable preserves WAN IP address. I suspect that a reboot of the router does as well since the modem retains power. But turning off my internet over and over throughout the day creates other problems that I have to limit. So much so that I really wish the 'block internet access' switch would work to cause the issue. But your suggestions does make me think that there is something there. So my Polisy has a reserved IP address. The Polisy is not set to a static IP internally, but the router always gives it the same LAN IP address. But if I 'block internet access' for just this device, the LAN IP address remains constant. But this should also be true if I remove power to the ONT. The LAN never goes down in that case. I am still unable to reconcile the difference between Data points we have so far: #4 and #5 I feel like there is a piece of information here that if I could isolate it might inform people smarter than me.

There is new OS for the Polisy coming out soon. UD and I plan to install that and continue testing, but Michel has low confidence that it will correct this particular issue. We were focused on hardware right up to the point that I can make it stop by severing the connection to the UD Portal. That is where we kinda hit a wall. Unfortunately, I do not have other hardware that I could switch to, and that would be a major endeavor. Our current plan is to install the new OS soon and if the issued continues do some network traffic monitoring. But I started this forum post because people much smarter than me do not think that is going to fix anything.

I have lots of Insteon, a little bit of Z-wave. I have the Zmatter board. Power supply is an APC DC UPS (CP12142LI) https://www.apc.com/us/en/product/CP12142LI/network-ups-12vdc-lithium-battery19500mah-bms-4led/ I have considered altering the power supply for testing, but I cannot bring myself to make the effort because IoX does not reboot unless the portal is connected and I loose and restore internet. I cannot see how a different power supply would alter that variable. TCP/IP is used extensively in my plugins and networking module.

Ooh, that is a good one! I like it. But alas, no change. IoX still reboots. I turned off "System Notifications", "UD Communications", and "Plugin Notifications" for all groups/devices. I also stopped the Notification Plugin for this test. Yes. I kinda have to. I did not use to, because my Polisy and my IoX programs are built to never restart unless I actively and manually do it. I have over 300 nodes, so it is a long time to 'Query All' if I am working on something that requires a reboot or restart. My Polisy is on a DC UPS, so does not loose power unless power is off for a long time. But now that my ISY restarts an average of once a week or so, I have to have 'Query at Restart' enabled or my ISY does not know the status of any of my devices. My whole wake up programs fall apart in this case. Disaster! I really feel like we can move away from the program angle. Disabling ALL programs does not stop the behavior. I admit to being a somewhat poor program writer, but I have been pretty great about never creating any kind of looping program. Surely UD would have seen that in the logs as you suspect. I like your thinking that the connection to a wifi device on the local network is involved, but the only thing I have like that is my Tempest weather station and associated plugin. But I feel like that is excluded by the stopping PG3x test. Further, my ISY does not get the weather info from the internet, it gets it locally from the LAN. And even if it did, I feel like the severing of the connection to the portal causing a change in behavior would exclude that since it has nothing to do with the portal.