johnnyt

Looking to deploy Polisy/IoP in another location and am looking to confirm/correct an assumption I'm making. I'm thinking I'd be able to connect one or more others devices to one or more of the 3 ethernet ports on a Polisy PRO such that the other devices can talk to Polisy/IoP *AND* the rest of my network plus the internet, which would all be upstream over the Polisy Pro's wifi connection. If the assumption is correct, will the Polisy Pro be acting like a router - assigning a different subnet to my other connected device and NAT'ing - or a switch/hub? Am I able to choose?

@bpwwer No, I didn't miss your mention of Tempest hub feeding the data to NS every 60 seconds. I just thought there might be the ability in the NS to control when it sends the data to ISY, i.e. store the hub data in NS table then every x seconds (or whatever user sets) potentially with y seconds delay in between send the data to ISY. I don't know what's possible and not possible so I'm just asking. Sorry for my ignorance. Didn't mean to offend or annoy. I'm hopeful the move to IoP will significantly improve things. I am, however, preparing myself for the possibility that I may need a second HA controller. I'm sure separating my HVAC from my lighting would solve my problems even if I stayed with the 994i but two controllers does have several drawbacks and added costs. Thanks for all the info you've been patiently providing.

Darn, I changed the short poll on Weatherflow from 60 to 150 thinking it would delay updates to every 2 1/2 mins but I now see it didn't have the anticipated effect. Any chance you could change that @bpwwer? Attached are sample log entries for both Weatherflow and Airthings for about the last hour. I downloaded the PG3 log, which only includes includes today's entries and I searched it for Try: 2 and Try: 3. Only found one Try: 2. but all the tries (Try: 1) I visually scanned for are taking hundreds and even thousands of milliseconds. Here's a sample. 8/15/2022, 24:00:18 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 448.735586ms - 8/15/2022, 24:00:19 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 723.250318ms 8/15/2022, 24:00:19 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 1021.064203ms - 8/15/2022, 24:00:25 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 703.886677ms - 8/15/2022, 24:00:25 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 743.913189ms - 8/15/2022, 24:00:26 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 1055.351757ms - 8/15/2022, 24:00:26 [pg3] debug: ISY Response: [Try: 1] [00:21:b9:02:55:cd] :: [200] :: 1103.256553ms - Couldn't attach PG3 log here I think because it exceeds the max for an attachment (it's about 1.5M compressed) but if it would help, I can grab a chunk and provide it. WeatherflowAirthingsLogEntries-sample.zip

So I can see - and I think Bob explained - that all NS events go into a single PG3 queue. That could certainly help manage the push of data to IoP but do IoP events also end up on that queue? If not, IoP could be really busy handling it's own events then have PG3 bombarding it with a steady stream of data (which trigger more ISY events - which is the problem and not the processing of getting the data itself) for extended period of time with no breaks or slow downs, no? I recently turned the Weatherflow NS back on after turning it and ST-Inventory OFF due to multiple ISY overloads to the point of crashing/rebooting when I first moved to PG3 and added it and ST-Inventory (which I now leave OFF), and Airthings NS'. See this post for all the gory details Since I've turned Weatherflow back ON I noticed most of my ISY lighting events are delayed, e.g. motion sensor that triggers lights takes forever to do its thing. Weatherflow is a very chatty NS and also takes 10-12 seconds to update all the data (not blaming the NS for all the data). I just saw an opportunity thanks to multiple Airthings devices to give ISY a break for what is my second most chatty NS. If the Airthings NS needs to call for each device's data one device at a time anyway, that would be an easy place to put in a break, I would think. That said, I get the point that if everything ends up on the PG3 queue and there was a lot of data on the queue, a break might not make a difference. Perhaps that means I need to ask Bob to consider adding a configurable "throttle" for PG3 queue processing? Of course that's not good for things where quick responses are desired. So then we would need to get into message priority setting by NS. The idea of putting a delay in Airthings NS was to make it kind of run at a lower priority, or at least lower speed. For sure I don't know how PG3 and IoP work, separately and together, so I'm no doubt missing a fair amount of the details that may make my suggestions completely 'out to lunch' but am throwing them out there anyway in case they lead to some tweaks for folks like me who love their ISY and have/want to give it as much work to do as possible. I've got plenty more ideas I want to throw ISY's way. I'm just waiting to be able to move to IoP so I can get past the 1000 program limit of the 994i. I just hope I won't be overloading IoP too.

Hi, @JimboAutomates, would it be possible to provide a configurable delay between device reports to ISY? Meaning NS would poll Airthings and send data for each device (6 sensors) to ISY with a configurable delay between each device. I have 7 devices and the update process can take 12-13 seconds under normal circumstances. An eternity in computer terms. I only request updates every 4 mins (what I think should be the default, BTW) to avoid exceeding Airthings rate limit and because updates to the cloud from each device are 5 minutes apart (though not all at the same time, of course). Technically that's 42 sensor readings. I can see from scanning my log files that not all 42 sensors typically end up being updated every 4 mins but most queries update 30 or more sensor readings. Now I do have an overloaded ISY (with close to 1000 programs) and might be the only one, or one of a few (which really surprises me) but please consider adding this in anyway next time you "open up" the NS. The data does not have to hit ISY in real time and allowing ISY to do other stuff in between device readings, if needed, is generally a good idea, IMO. I'm also in the process of planning an Indoor Air Quality automation package to resell a combination of Airthings sensors with a pre-programmed Polisy/IoP and your NS to capture data and automate HVAC functions. Am just waiting to see if the next few months lead to major improvements in terms of ISY Zwave support (and Matter, I'm reading) and, for my own use, a long awaited zwave migration tool from 994i to IoP to confirm a number of things.

yep. use it all the time and do like it. I suppose I could open it up on the side while I'm doing Admin Console related stuff with the Tempest weather station. It's the shear number of fields that comes with the Tempest weather station that's the problem, which is not the worst problem to have.

oh yes, right. Here are my current numbers from yesterday's run:

I caught the following in the event viewer: Sat 08/13/2022 11:41:54 AM : [ n006_207296] WINDDIR 265 (uom=76 prec=0) Sat 08/13/2022 11:41:54 AM : [D2D*CMP 036F] STS [n006_207296] WINDDIR B Cannot convert values (from=0E to=4C)

I have network resources but none related to this NS. I also don't have any programs related to it. I just start it, read the info (mostly number of programs as I'm near the maximum allowed) then stop it. That said, if you want to more info on my network resources let me know what info you'd like. Don't worry too much about this - at least for me - as it's not affecting how/what I'm using the NS for. Just wanted to mention it for your awareness. Thanks

Looks like it. Am on version 1.08, which is the version the Node Server Store shows.

Figured that might be the case but asked anyway in case there was something that could be done.

Just caught a "West" report for wind direction that when back to degree on next update. and the variable I try to set refuses to change (stays at 0)

just noticed the following error come up in log: 2022-08-12 12:59:02 error: NS: Parsing Error: TypeError: Cannot read properties of undefined (reading 'includes') at _0x435040.processNetworkResources (/var/polyglot/pg3/ns/00:xx:xx:xx:55:cd_7/Nodes/ControllerNode.js:1:5971) at _0x435040.onDiscover (/var/polyglot/pg3/ns/00:xx:xx:xx:55:cd_7/Nodes/ControllerNode.js:1:3118) I've attached the log for today. If you're wondering why there's not more data it's because I only run ST-Inventory manually once in a while to reduce performance issues I have with my ISY (not powerful enough for the almost 1000 programs I have) and asking for all the info is taxing. Could this error have been because ISY was overloaded when it checked it? ST-Inventory_8-12-2022_12205_PM.zip

for the record, I right click and select "Run If" to force the condition check. It does the same thing enabled and disabled.

Hi, is there a way to reduce the size of the field display boxes in ISY Admin Console GUI? As it is I can only see all the the data when using my 32" monitor set to 2560 x 1440 resolution. Below is what I see at 1920 x 1280 resolution. The following three fields are missing at the bottom. It only gets worse at lower resolution (that I sometimes need to live with when I VPN into a machine from elsewhere) Alternatively, is there a way to select what the fields in the bottom few rows are so one could move less interesting data lower down? Thanks.

I have a program that checks for wind direction >=0, which should always be true but is always false. Attached is screenshot of the program showing it's evaluated "false", screenshots of ISY GUI and NS Node value for wind direction. I downloaded the log (attached) but it had no data (at warning level) going back 3 days or so, which seems strange. so I changed level to "debug" and restarted NS, and attached that second log after letting things run for a minute or two. WeatherFlow_8-12-2022_103244_AM.zip WeatherFlow_8-12-2022_102852_AM.zip

Correct me if I'm wrong but when you say "sensor" in the issue explanation on github, I think you mean "device", right? I have 7 devices and each one has 6 sensors. Timing for short poll should be 30 X 7, not 30 X 42 (7x6). Is it sending data for all sensors in all devices or just the changed data since last short poll? It looks like it's only changes because I rarely see 42 messages being sent to ISY but I want to confirm in case the reason I'm not getting 42 updates every time is because there's a problem somewhere. For reference I set my short poll to 240 (4 mins). I think devices update the mothership every 5 mins, each on it's own schedule. I haven't seen any rate limit issues since I moved to 4 mins.

Thanks @bpwwer. You make a really good point about how all NS' are starting at once when Polisy/PG3 is restarted. That alone (along with programs it triggers) may overload my 994i, even if it has finished doing its own stuff at startup. I may try what you're suggesting but I'm pretty sure I'll find there's no problem if I simply turn stuff off. I think the situation is just that there's too much being asked of the 994i. Hopefully IoP will resolve that. For now I've turned off WeatherFlow NS as it can send a lot of data in quick succession and I haven't really begun to leverage that data as I just got the device and NS a few weeks ago. So far so good. Bottom line, though, it seems I'm firmly at a point where the only thing that works right now is to not automate stuff I'd like to automate. In the interest of making the Polisy/IoP able to handle problems when they happen (while the humans are at work, sleeping or away) as well as handle the workload demands of up to 2000 programs, I wonder if the following features could be added to PG3 and ISY, where applicable: (I know you don't control either but believe you have influence with the ones who do) ability for ISY to stop/start/restart individual node servers. Not just for working around problems, but also consider that maybe some NS data/functionality is not needed while sleeping or on vacation, while some is only needed when sleeping or away ability to stagger the startup of NS' using a configurable delay by NS/slot or as a single global setting. Ideally there would be a way to select NS startup order, but as a starting point just the ability to manage the bombardment of ISY at startup would be good. ability to throttle the messages being sent to ISY, e.g. make all NS' put stuff for delivery to ISY on a single PG3 queue (if that isn't already the case) and set a configurable delay between each item being sent to ISY. Retries would see the item being added to bottom of queue rather than compounding what may be a processing capacity issue. Ideally there would be a way to set a message (or NS) priority as some NS data is needed faster than other. Also, there would be a way to adjust the throttle via an ISY program. I think this feature could potentially negate the need for #2 as ISY would be able to adjust throttle to a very high delay between messages when it's restarting. To give a bit more context on my set up because my need for more processing capacity and better processing efficiency isn't going away, and because I've had a few people in the past politely suggest that surely better programming on my part would fix my problems. My biggest source of programs (estimated at 70-75%) are related to HVAC. Unlike lighting, there are many more conditions and data points affecting HVAC decisions/functions. It's easy to underestimate all the conditions. Believe me. Especially when there's huge differences in seasons where one lives (I'm in Eastern Ontario, Canada) For example, here's some of the data I have to get and act on: indoor and outdoor temp and humidity, forecasted temp (today and tomorrow), temp differences within the house, temp diff inside vs outside, humidity differences, time of day, season (based on temp, forecasted temp, and time of year), home vs. away vs. on vacation. (OWM provides data that's critical to proper functioning, which is why I'm looking for some self healing tools if things aren't working between it and ISY for whatever reason.) Aside from controlling heat, AC, furnace fan (low/high), HRV (low/high), HEPA filter (low/high), duct fan (low/medium/high), humidifier in winter, two dehumidifiers in summer, and 2 bathroom fans based on the above data, I also have 7 Airthings devices that I use to trigger ventilation when VOC or CO2 levels are high. I also have to deal with humans wanting to manually run something or change the thermostat mode/set point without the automation coming right behind and undoing it, yet eventually taking over again. It's way more complicated than you might think, and certainly than I expected when I started on the journey. Over past 12-18 months, I've tried multiple times (and am always looking) to reduce the number of programs. I've had some successes but not on scale that would be needed. Last year I completely scrapped and started over the chunk controlling my HRV to see if I could lean things up and reduce the number of programs. In the end I ended up with just as many programs, if not more, and probably 40+ hours of my life I would have rather used to add automation not fix what I already had. In the mean time, to cope with overloads at start up and minimize them at other times, I've added different 'wait' commands throughout most programs to help reduce concurrent load to some extent. Of course that's a bit of a guessing game when you have 900 programs. Also in the category of "hours of my life I would have rather used to add automation not fix what I already had", I've had to build a complicated startup sequence for things to work mostly reliably, otherwise I would get "queue full" messages and (unknown to anyone) actions would be dropped. Informal scan suggests it works 60% of the time and that, for the other 40%, there are only a few missed actions that seem to fix themselves later. I never had a startup crash for some reason (maybe because I delay Polisy start?). I've just had stuff not working properly after a startup (most recently the interaction with OWM NS.) Key for a large majority of my programs is that very few things need to be done instantaneously with HVAC and other stuff. A lot of it can easily wait a bit. Most lighting automation needs quick response but even some lighting can wait, e.g. outside light ON when it's dark. What's another 30 seconds when turning on outside light, HRV or Humidifier? I propose the ability to manage work with throttling and prioritization would allow for demand/supply imbalances to be handed gracefully for 1000 programs today and up to 2000 programs eventually. It feels like some of that is already in place within ISY, e.g. backups clearly run at lower priority, but not with respect to the crucial interactions between PG3 and ISY. Personally I think HVAC automation is the next big thing in HA, and HVAC enthusiasts (nerds) like me don't mind and even like programming an ISY, so for the benefit of the ISY and the UDI model, I'm really hoping Polisy/IoP will be able to handle 2000 programs, especially when a majority doesn't need to be processed instantaneously in the interest of being able to do the stuff that does need rapid response and allow for everything one wants to automate. I mention all this believing you have influence in what PG3 (and possibly ISY) is/will be capable of doing, but if I'm barking up the wrong tree, let me know. Thanks again.

For Chrome (my usual choice of browser) I tried closing and re-opening it and the uptime was the actual number. For fun I went to Firefox and got the message that PG3 had be up > 3 months. I assure you I haven't had firefox running for 3 months straight. Heck I've had Windows do update and a restart a couple times in that time. But after I cleared the cache, the actual uptime came up and there was no longer message about PG3 needing restart.

So after investigating and capturing the above info, I went ahead and restarted just the OWM NS and all is fine now. This is good example of how an ISY initiated NS restart function would eliminate the need for the "sledge hammer"/ IoP suicide mission of doing a Polsy power cycle, which actually didn't fix the problem this time (probably my ISY missed something at restart despite the 10 min delay in restarting Polisy) Maybe when I move to IoP I won't have these issues but no one is telling me that (including UDI) so it remains to be seen. Even then, will I be limited in what else I can ask ISY to do for me, if not initially then 2-3 other node servers and 200-300 programs down the road? I won't have the luxury of delaying start of PG3 when I move to IoP unless I buy a separate Polisy just for IoP...

Just for reference the Polisy power cycle is something ISY does when it reboots, and it waits 10 mins after turning the (WebSwitch outlet) for Polisy off before turning it back on so the ISY has time to stabilize. I had to build complicated, multi program startup routine to do things is chunks otherwise I get "full queue" messages and a variety of things don't happen as they should. My startup routine works 60% of the time, and for the other 40%, I think I've been able to minimize the things that don't happen. I digress a bit to provide some context. The problem right now is that the data in ISY has not updated since before the spontaneous reboot at 11:53 (about 7.5 hours ago). Here are the variables that get updated when ISY sees changes in values such as temperature and humidity Here are two programs that should have set these variables to something that's not 0. There are others too that aren't working. Set Current Humidity - [ID 0303][Parent 0330] If '1-HVAC and Backyard / Climate / OpenWeatherMap' Humidity > 0% Then $sHum.Outdoor.Weather = '1-HVAC and Backyard / Climate / OpenWeatherMap' Humidity % Else - No Actions - (To add one, press 'Action') Set Current Temp Weather Service - [ID 00B1][Parent 0330] If '1-HVAC and Backyard / Climate / OpenWeatherMap' Temperature > -50.0°F Then Wait 3 seconds $sTemp.Outdoor.WeatherService = '1-HVAC and Backyard / Climate / OpenWeatherMap' Temperature °C Else - No Actions - (To add one, press 'Action') Interestingly when I saved data from the ISY event viewer and separated out the data for OWM there are regular updates being seen by the ISY. See attached spreadsheet. However the displayed data has not changed accordingly (and shows node server is not online) and even if I run the program for temp update manually, it doesn't change the variable. The program is "True" (green) but either the data is 0 or ISY is badly broken and the program isn't actually performing the action. Here's data right now in ISY that I believe is from before 11:50 this morning. This is very strange. OWM_EventViewerData.zip

Hi, I had an ISY 994i lock up last night so power cycled it and Polisy to recover from that. Recovery did not go so well for OWM NS. I've attached today's log file. ISY lock up occurred at or shortly after 01:33:15 AM last night and was power cycled at 09:53 AM. Polisy turned off as well around 9:53 but only turned back on at 10:10 to let ISY settle. One of my ISY programs subsequently power cycled Polisy around 11:00AM because OWM had not updated ISY data for too long. It was still seen as disconnected as I started writing this but a subsequent ISY program initiated power cycle (about 11:44) fixed it. (As you know in another post I discuss needing ability to restart NS or PG3. This is an example of why it's needed, at least until all NS' can detect and recover from ISY lock ups, restarts and other issues. This weather data affects multiple HVAC related program decisions so I need to make sure the data is fresh enough. I do get notified when these Polisy restarts are called so I can look into them but, until I can fix the root cause, the show must go on and a self healing mechanism needs to be available because I can't always fix issues as soon as they occur. When I move to IoP, these power cycles will become suicide missions for ISY, which is really not ideal) OpenWeatherMap_7-21-2022_115802_AM.zip

I power cycled Polisy to recover from ISY994i lockup and, for some reason, the dashboard reports PG3 has been up for > 5 days. I've attached today's log file. ISY lock up occurred at or shortly after 01:33:15 AM last night and was power cycled at 09:53 AM. Polisy turned off as well but only turned back on at 10:10 to let ISY settle. One of my ISY programs also subsequently power cycled Polisy around 11:00AM because one of the Node Servers had not updated data for too long (a story for another post, maybe. I have to look into that) pg3_7-21-2022_110421_AM.zip

While Java is far from being a real time OS, I find it hard to believe the AC running on a modern CPU is a bottle neck in the AC's loading performance. Just as interesting are backups, which can take well over 6 minutes. Fortunately (though not for the time it takes) I can tell the backup occurs on a low priority because the time it takes varies greatly. This would help explain why I've never seen a crash doing one of those, and I think I would have most certainly seen one by now otherwise. Data wise a backup in my case is 632 KB of uncompressed data (413KB compressed). I don't know how much of that are programs for a straight comparison, but at 6 minutes (when things aren't too busy otherwise) that's 1.7 KB/s throughput, or about equivalent to 14.4 kbps dial up modem speed. So I'm not seeing the problem as related to the amount of data. While I confess I have no idea how things are working in the background, it's really hard to believe my i7 core CPU with hyper threading (4Cores/8Threads) running at 3.4 GHz (turbo up to 3.8 GHz) connected via Gb Ethernet wouldn't be able to support the AC processing waaay more data per second than that. While I'm pretty sure the Polisy does not have close to an i7 core, I was really hoping that both CPU and network speed improvements of the Polisy/IoP were going to mean a HUGE improvement in overall data processing speed - with 2 aspects to this: 1) IoP loading the console, and 2) IoP talking to NS'. Does/will the console loading not benefit from Polisy hardware improvements? Could the first 300ms to load the first 88 programs largely be attributed to the "startup" processing and end up notably more efficient for the next 900, or 1900? Please at least tell me IoP is talking at something approaching internal bus/memory speed to node servers (minus PG3 overhead) and that it will make a noticeable difference? One of my ideas has been to put my lighting related insteon on one ISY (either 994i or IoP) and, on the other, put zwave with some insteon (all HVAC related). I'd rather not have to do that but if I'm deluding myself thinking that Polisy/IoP will fix my issues, I may go there. Your advice would be welcomed.

Hi @simplextech I've decided to leave ST-Inventory off for now after discussing my crash issues with UDI. You'll find the rationale at the post below after I got into talking about ST-Inventory with @bpwwer.