automatewhatican

Interesting. Another lock-up occurred overnight. I haven't rebooted yet. I need an escalation plan; the repeated hard fails are not sustainable. Does anyone have a suggestion? I will pull a ticket, referencing this thread. The display tail of error messages this morning on console seems the same last two times:

It's in a computer closet with a reasonably controlled environment. Current ambient is only 81.4F. eisy itself feels warm to the touch, not hot.

On plugins for this eisy, I'm running production versions of OpenWeatherMap 4.0.3 and iAquaLink 3.1.10 plugins and have a production version of Notification 3.6.32 that is UNconnected in PG3. On the ISY side, I have a pretty good set of Z-Wave and Insteon nodes, probably at least 20 of each, although I can do an inventory if helpful. Zwave is mostly AeoTec multisensors and switches and a home energy meter node that has been unpaired for a couple years but for which an inactive node remains. I also have three Honeywell T6 thermostats and a Yale lock. I have about 20 programs; nothing super complicated, but a fair amount of environmental monitoring, and motion or time-cued lighting and thermostat setpoint controls. Insteon is entirely lighting controls. Other than the standard 3am Query and any usual eisy/freeBSD housekeeping, I don't think there is anything particularly noteworthy in the overnight execution. Z-Wave is an original ZMatter; I've never updated ZMatter firmware. Other than that, packages are all pretty current. Does anything catch your eye as a good line of inquiry/investigation?

Overnight, memory utilization (as indicated in eisy-ui) spiked to 94%. Less than 0.5 GB remains free per system monitor (189M Active, 700M Inact, 7812K Laundry, 6270M Wired, 465M Free). Nothing strikes me as inherently problematic in that, but it might be notable that someone at UD decided to code the thermometer bar to present in RED rather than BLUE at these levels. Is there something else I should be checking? ... or just wait for lock up and then pull some log?

Will do, Guy. So far so good; the memory utilization (as indicated in eisy-ui) has been hovering around 25% and about 6GB of free memory is showing in freeBSD system monitor. Agree on unusual. Reliable, non-stop unattended performance is really the isy value proposition, but these recurring lockups are now of concern. I will give it 48 hrs, checking memory; failures appear to occur in early AM. If there is anything else you'd suggest monitoring or checking, let me know.

Thanks, Guy. I understand the up pressure from memory paging and caching. The real problem is the repeated lock ups, requiring hard restart. The eisy-ui's RED symbology for the 95% utilization is what caught my attention. Next time, I'll check the proportion of the pool containing active pages, leaving aside dirty inactive pages.

Done. As expected, down to 24% (1.8 GB / 7.7 GB). I guess we'll have to see if it creeps back up over the next 36 hrs.

I've recently had two lockups of the same eisy since updating to 14.4-RELEASE-p6. The lockups were roughly 48 hours apart. In each case, the console showed a controller timeout and register dump together with an indication of an unrecoverable I/O failure in the ZFS pool. See below: After reboot, I ran a zpool status, which reported no errors. It's now been about 36 hours since the last lock up, and I noted the following in eisy-ui: Is there possibly a memory leak in IoX 6.1.2_7? What do you suggest?

Thanks for the update, Javi. And thanks for the thorough problem summary. Is there a way to provision a portal/proxy connection on an ISY944 running 5.3.4, but which is not current connected through the proxy? For some reason, I do not get an opportunity to approve a portal connection for the affected uuid. I also do not get the Configuration | Portals tab in Admin Console. If we could get communications with the ISY944 established through the portal/proxy that would be a useful workaround until the remote device can be physically upgraded to (replaced with) an EISY.

I have this same issue. I have been able to resolve for an EISY deployment, but I have not been able to resolve using Javi's workaround for an ISY. For the ISY (running 5.3.4), no Portal tab ever presents, so the portal communications based workaround is unavailable. Help please.

Thanks dbuss. Were you seeing successful reception of Zwave message frames, but failure to transmit?

I don't think it's a USB issue as the failure has presented w/ and w/o a short (3ft) intervening USB cable AND when the dongle is plugged directly into various of the USB ports of the EISY. I'll pull a ticket.

Since transitioning from ISY944/500series ZWave to EISY/ZMatter, I have had a problem with ZWave communication failures that affect ZWave communication with my entire ZWave network (20-30 devices) is no longer controllable. The failure seems to present after a week or two of uptime and I've been "fixing" by rebooting and removing power cycling various components (including the EISY and ZMatter). Rather than power cycling, today I went looking for clues. I noted the following based on event logs and behaviors: outbound communications from the EISY/ZMatter show failure (in logs). This is consistent with the observable that device states do NOT change based on commands initiated at UDMobile or admin console. inbound communications do seem to be being received, even when the ability to control ZWave devices is down, and mirrored states are generally being updated based on states changed at ZWave devices. during normal operation, I would expect periodic and somewhat frequent (once a minute) telemetry from an Aeotec ZW095 Home Energy Meter. However, during the ZWave down condition, I'm not receiving those. Today, rebooting EISY and removing/restoring power to the ZW095 seemed to resolve the problem. For years my ISY944 was rock solid with the network of the same ZWave devices. This period human-in-the-loop reboot that I'm now experiencing is not going to work. For a while, I just assumed that a forthcoming IoX release would fix; however, the failures remain after updating to 5.8.3. Any ideas? How do I open a ticket? ... and what other data should I try to collect?

Yes, on the z-wave alliance site, "Z-Wave Plus Version 2" signifies a series 700 device.

Resolved with Pro upgrade. I concur with @MrBill, non-Pro node limit is easy to hit, particularly with some of the newer Z-Wave devices that result in many subsidiary ISY nodes. Based on a quick manual count, I’m carrying 30 physical devices that are Z-Wave and 22 that are Insteon (including 5 KPLs).