Home Assistant ISY Component

[jec6613]

8 minutes ago, simplextech said:

BTW I'm a Unix Engineer by trade not a developer. As a Unix guy I learn what I need to as needed.  Now I'm confused.... You're looking at HASS even though you say it was not performant for you?  Or are you only looking at HASS for the UI only?  I ran HASS for the UI integration and side-by-side systems and the ISY for a while and then HASS updates broke this and then broke that and then it was good then broke again... and again... and... I moved on.  I still have it installed to play with but it doesn't actually do anything or really even display anything... but it gives me ideas for Nodeservers to write

I'm looking at CQC for the UI aspect and to tie in other things that I can't integrate nicely with ISY.  I'm still using the ISY and Nodeservers and keeping the ISY as CQC does not replace that it sits at a layer above device control and is the orchestrator to "glue" pieces together into a consistent interface.  I don't think it requires being a developer to understand but it does help

Exactly, HASS for the UI only.  It works well for that, it's the automation and sorting through and issuing dozens of commands in quick succession that was the problem for me, it could take 10+ minutes to execute what an ISY can do in about 5-10 seconds.  I'm going to sort out the update issue the simple way ... it's a VM so I can just checkpoint it.

I've looked at CQC in the past, but it's spendy for what it does and doesn't offer the flexibility of new interfaces spun up so quickly or the sort of management flexibility of a web app.

As a Windows guy I use very different tools, and the learning curve to development is much steeper.  On the other hand, I have lots of fun OS-level things to do what I need, Windows Server really includes a ton of extra stuff for free with it, from RADIUS and PKI, to proxies and VPN, and it's all well documented and works very well and doesn't require hacking things together.  The downside, of course, is I'm expected to be an expert on about 1,000 different ways to use a Windows server.

Edited January 10, 2020 by jec6613

[simplextech]

2 minutes ago, jec6613 said:

Exactly, HASS for the UI only.  It works well for that, it's the automation and sorting through and issuing dozens of commands in quick succession that was the problem for me, it could take 10+ minutes to execute what an ISY can do in about 5-10 seconds.  I'm going to sort out the update issue the simple way ... it's a VM so I can just checkpoint it.

If any event/action takes longer than 1-2 seconds I'm annoyed up to the 3-5 seconds and I'm throwing it out. 

4 minutes ago, jec6613 said:

I've looked at CQC in the past, but it's spendy for what it does and doesn't offer the flexibility of new interfaces spun up so quickly or the sort of management flexibility of a web app.

It's pricey but quality generally is.  Free is what it is and like you said you can checkpoint and restore and all is well.  I prefer not having to babysit which is also why I like the ISY.  The CQC UI builder is easy to stat with using the Auto Generated via templates.  It's Windows point and click.... add a light, media, hvac, to a room look through devices select them and click the generate button and wait.... done.  Nice interface template that you can then with point and click editing change around and be as creative as you want... did I mention point and click? 

Auto gen just for example:

That's running in my web browser (FireFox) so it will run on essentially any device.  Yeah that's a Concrete Blonde album that's in the music library.  All of that was auto generated no code.  I clicked on the 'select room' at the bottom left to get the popup in the middle of the screen by the way. 

I'm still just tinkering and toying.  I've been submitting bugs to Dean at CQC who has been fixing them left and right.  UDI is the only other company that compares to customer support and quality that I know of.  This is making me want to support both and throw money at Dean to keep developing, improving, extending CQC.

[jec6613]

Well I was able to patch it and get it to pull in info to the ISY, but it appears that HASS itself is bugged and main processes crash with out of memory errors.  Which I didn't think was really possible running any sort of supervised code like Python, and the VM's memory load isn't nearly high enough to trigger a physical out of memory.

Worse: when it comes back up, it turns tons of things on.  Not an Insteon All On event, but every LampLinc comes on, which isn't good.

[simplextech]

32 minutes ago, jec6613 said:

Well I was able to patch it and get it to pull in info to the ISY, but it appears that HASS itself is bugged and main processes crash with out of memory errors.  Which I didn't think was really possible running any sort of supervised code like Python, and the VM's memory load isn't nearly high enough to trigger a physical out of memory.

Worse: when it comes back up, it turns tons of things on.  Not an Insteon All On event, but every LampLinc comes on, which isn't good.

Sounds like a good time to me.... I'm happy exploring the land of CQC

 

[carealtor]

1 hour ago, jec6613 said:

Well I was able to patch it and get it to pull in info to the ISY, but it appears that HASS itself is bugged and main processes crash with out of memory errors.  Which I didn't think was really possible running any sort of supervised code like Python, and the VM's memory load isn't nearly high enough to trigger a physical out of memory.

Worse: when it comes back up, it turns tons of things on.  Not an Insteon All On event, but every LampLinc comes on, which isn't good.

That's so weird.  I have no issues what so ever.  There has to be something in you guys's set up that I don't have.

[rccoleman]

Just now, carealtor said:

That's so weird.  I have no issues what so ever.  There has to be something in you guys's set up that I don't have.

I've had the same feeling watching the back-and-forth on this thread.  The isy994 custom component from HACS and the built-in one before that both worked fine for my fairly extensive Insteon installation that supports all the wall-switches in my house.  I still see retries triggered every so often when the ISY erroneously reports "404", but it works after a few seconds.  I have removed all nodeservers from my ISY now that I'm controlling all of my third-party devices through HA, so my ISY essentially operating as a glorified Insteon hub may help avoid some hiccups.  But my experience is that the ISY994 component and HA itself has been pretty rock-solid for the last 6 months or so.

[simplextech]

11 minutes ago, carealtor said:

That's so weird.  I have no issues what so ever.  There has to be something in you guys's set up that I don't have.

It's either some device or some scene.  I have two other ISY's that I've tested against.  One with some nodeservers and one with NOTHING in it and the component loads fine.  But an ISY with no devices is not much use. 

I already tore my whole system down to zero nodeserver and removed z-wave and still it's broken... I'm not going to go and start deleting scenes and insteon devices one by one hoping to find the unicorn.  Not when I have alternatives that just work and have support.  I had a couple issues and Dean from CQC fixed them fast.  I have a few requests that he's accepted but he's busy on a major code overhaul so I'll have to wait... that I can accept.  Silence and ignoring a problem I can't.

[jec6613]

  

5 minutes ago, rccoleman said:

I've had the same feeling watching the back-and-forth on this thread.  The isy994 custom component from HACS and the built-in one before that both worked fine for my fairly extensive Insteon installation that supports all the wall-switches in my house.  I still see retries triggered every so often when the ISY erroneously reports "404", but it works after a few seconds.  I have removed all nodeservers from my ISY now that I'm controlling all of my third-party devices through HA, so my ISY essentially operating as a glorified Insteon hub may help avoid some hiccups.  But my experience is that the ISY994 component and HA itself has been pretty rock-solid for the last 6 months or so.

I think it's just a straight up size issue.  Nodeservers eat up a lot of memory on HA, and if I had to guess a single add-on is just chewing up too to load well over 1,000 items.  The thing is though, I'm not moving anything out of the ISY because every nodeserver and everything else does do stuff.

I've run into HA performance problems before though, when I tried to see if it could run my Insteon setup.  It had no chance then, either.

It could also be that HA is primarily designed for ARM and the x64 code is a bit of an afterthought.

[simplextech]

6 minutes ago, rccoleman said:

But my experience is that the ISY994 component and HA itself has been pretty rock-solid for the last 6 months or so.

It used to be very solid.  I have no idea what has changed or broke it.  But I can't tear my home apart trying to find what it could be and with Insteon and the ISY I don't have a spare serial PLM to use.

Over the weekend I went as far as to backup my ISY and I actually did remove all of the scenes from the test ISY... and the component still would not load.  So it has to be some device in my system but I don't have any idea what it could be.... I even backed out firmware levels and still had issues.  Maybe I'll try again after I'm done playing with CQC but by then I don't think I'll have a reason for HASS.

[mwester]

Just now, jec6613 said:

  I think it's just a straight up size issue.  Nodeservers eat up a lot of memory on HA, and if I had to guess a single add-on is just chewing up too to load well over 1,000 items.  The thing is though, I'm not moving anything out of the ISY because every nodeserver and everything else does do stuff...

Kinda proves my point -- the concept of "Let's duplicate EVERYTHING from the ISY into HASS!" is flawed right out of the gate.  We'd never to that in any other system architecture, so why do it for the ISY?

(I think the answer to that question might perhaps be in the way the ISY is designed -- you either grab info bit-by-painful-bit, or you subscribe to the "fire-hose" that sends every action about everything in XML down the wire.  Of course the receiving side could filter that - but filtering is hard, and doing it on the RX side is not the ideal place for that either...)

If you have 1000 nodes on the ISY, it makes no sense to mirror that into any other system (and it won't matter if it's x64, arm, or an IBM mainframe!)

[simplextech]

1 minute ago, jec6613 said:

  

I think it's just a straight up size issue.  Nodeservers eat up a lot of memory on HA, and if I had to guess a single add-on is just chewing up too to load well over 1,000 items.  The thing is though, I'm not moving anything out of the ISY because every nodeserver and everything else does do stuff.

I've run into HA performance problems before though, when I tried to see if it could run my Insteon setup.  It had no chance then, either.

It could also be that HA is primarily designed for ARM and the x64 code is a bit of an afterthought.

It's not a ARM vs x64 issue because it's all Python.  Python doesn't care too much about the underlying system.  However Python is not the most speedy of code in large globs like HASS.

One thing to be aware of is that HASS actually is very heavy in the way it handles components.  A portion of all those 1000+ builtin components gets loaded on startup of HASS regardless of them being used or not.  They are part of the base.  Even though they are small pieces they really add up.

A couple years ago I hit the size limit with HASS which led me out to HomeSeer and other systems because of size and limits.  There's a size limit with the ISY too except they tell you upfront what it is and it's based on the ISY and PLM limitations.  Polisy eventually will help this on the ISY limits (CPU/Memory/Net) but it's still a single system with finite resources the same as any other monolithic system or hub based system.

[simplextech]

Just now, mwester said:

Kinda proves my point -- the concept of "Let's duplicate EVERYTHING from the ISY into HASS!" is flawed right out of the gate.  We'd never to that in any other system architecture, so why do it for the ISY?

(I think the answer to that question might perhaps be in the way the ISY is designed -- you either grab info bit-by-painful-bit, or you subscribe to the "fire-hose" that sends every action about everything in XML down the wire.  Of course the receiving side could filter that - but filtering is hard, and doing it on the RX side is not the ideal place for that either...)

If you have 1000 nodes on the ISY, it makes no sense to mirror that into any other system (and it won't matter if it's x64, arm, or an IBM mainframe!)

I have a 20K+ core HPC at work... I think I can handle it

[rccoleman]

I'm running HA on a mostly-dedicated I5 with 16GB of RAM and total usage is usually in the 5GB range.  While many do run HA on a Pi, I got tired of that really quickly and I can see where you'd run into resource issues.  I had NodeLink and 5-6 Polyglot node servers when I started messing around with HA last year and I certainly wasn't up to that number of additional nodes.  When I decided that my general direction would be towards HA and every nodeserver that I used had a native analog there, I switched to the HA version and left the Insteon devices and scenes on the ISY.  On the plus side, the ISY admin console loads much, much faster now

Edited January 12, 2020 by rccoleman

[simplextech]

There are merits and benefit to this.  However HASS using that much RAM is worrisome.  I'm going to guess you have all of your automations in HASS as well?  I wonder how quick they are to respond to changes with that much happening in HASS.

[rccoleman]

Yes, all automations are in HA, either through YAML automations/scripts or through Python Appdaemon scripts.  The only perceptible delays are due to cloud-based services taking longer to respond or the ISY periodically pretending that it doesn’t have certain ISY devices or scenes and returning a 404. Honestly, unpredictable timing of program/device response on the ISY was one of the things that pushed me to experiment with HA and I find HA to be much more predictable.  It’s also leaps and bounds easier to debug when strange things do happen.  It’s at least as fast as the ISY, but I’m also bringing considerably more processing power.  It’s pretty close to instantaneous to respond for triggers and actions that are under its control.  

I just looked at the resources in use and the active memory was 3.6G for HA and a Plex server together, with a load avg of 0.4 on a 4-core i5. It’s more machine than HA needs, but it was only about $300 all-in for a refurb Lenovo m92p tiny PC form factor. 

[mwester]

I moved my hass system from the rpi to one of my kid's retired laptops - not because it was slow, but I had an SD-card failure on another rpi that convinced me that platform's I/O isn't ready for something that read/writes to disk a lot and needs to run 24/7.

However, I just successfully got a core (base) hass system up and running in a "jail" (FreeBSD's answer to docker) on the Polisy... that probably won't scale as large as an i5, that's true - but for small environments, that has a lot of potential.

[simplextech]

30 minutes ago, rccoleman said:

Yes, all automations are in HA, either through YAML automations/scripts or through Python Appdaemon scripts.  The only perceptible delays are due to cloud-based services taking longer to respond or the ISY periodically pretending that it doesn’t have certain ISY devices or scenes and returning a 404. Honestly, unpredictable timing of program/device response on the ISY was one of the things that pushed me to experiment with HA and I find HA to be much more predictable.  It’s also leaps and bounds easier to debug when strange things do happen.  It’s at least as fast as the ISY, but I’m also bringing considerably more processing power.  It’s pretty close to instantaneous to respond for triggers and actions that are under its control.  

I just looked at the resources in use and the active memory was 3.6G for HA and a Plex server together, with a load avg of 0.4 on a 4-core i5. It’s more machine than HA needs, but it was only about $300 all-in for a refurb Lenovo m92p tiny PC form factor. 

HASS + Plex with the memory usage makes sense.

I've also found that there's some spurious 404 errors out of the ISY and it's in how the client side is handling the subscription to the ISY.  This is something I debugged with Dean from CQC when I first started messing with CQC.  I'm not exactly sure what was done on the CQC side as I don't have access to that code but the polling rate for the driver was increased for for the async nature of the connection.  Or at least that's all I was told.  I think PyISY needs some updates in this area as well to handle this.

However I also understand there are still other inconsistent timings that I have just with the ISY and programs where sometimes things are very fast in execution of the program and other times very slow 2+ seconds up to 5 seconds or they just don't happen.  There's no log or anything which is very frustrating.  In an attempt this last week to try and see why things were sometimes slow I removed all nodeservers and all z-wave devices that were noisy just to try and see if I could find something in the event log.  Nothing.  so I have no idea why sometimes things are fast and slow other times and why sometimes a light just doesn't come on when it's supposed to.

I even replaced a switch this weekend from a suspicion it may be the switch.  It was an older KPL that none of the buttons were in use anymore so I replaced it with a new switch.  I'll see if that garage light not turning on "sometimes" is resolved from the hardware change.  I don't know. 

Yeah for $300 that's a good piece of kit for running HASS on. 

[simplextech]

6 minutes ago, mwester said:

running in a "jail" (FreeBSD's answer to docker) on the Polisy...

BSD jails and Docker are nothing alike and "jails" have existed forever and are closer to a "chroot" environment.

[mwester]

1 minute ago, simplextech said:

BSD jails and Docker are nothing alike and "jails" have existed forever and are closer to a "chroot" environment.

Yeah I get that - just trying to provide readers of the thread with something somewhat familiar.  Sorry to bother you!

[simplextech]

Just now, mwester said:

Yeah I get that - just trying to provide readers of the thread with something somewhat familiar.  Sorry to bother you!

Not bothering me.  jeez.  I'm a unix geek I'm picky ok.

 

[rccoleman]

36 minutes ago, simplextech said:

HASS + Plex with the memory usage makes sense.

I've also found that there's some spurious 404 errors out of the ISY and it's in how the client side is handling the subscription to the ISY.  This is something I debugged with Dean from CQC when I first started messing with CQC.  I'm not exactly sure what was done on the CQC side as I don't have access to that code but the polling rate for the driver was increased for for the async nature of the connection.  Or at least that's all I was told.  I think PyISY needs some updates in this area as well to handle this.

The conclusion from discussions here is that the ISY responds with 404 when it’s busy or thinks the Insteon network is busy, so I contributed a simple retry mechanism that’s now in PyISY. There were also some serious issues in PyISY that could cause non-deterministic delays (basically not recognizing the response to some commands until the start of the next one), but that’s also fixed in the latest PyISY. It was dormant for a few years and has recently gotten some much-needed TLC, which is good for anyone using PyISY, including HA. Yay open source!

[simplextech]

1 minute ago, rccoleman said:

The conclusion from discussions here is that the ISY responds with 404 when it’s busy or thinks the Insteon network is busy, so I contributed a simple retry mechanism that’s now in PyISY. There were also some serious issues in PyISY that could cause non-deterministic delays (basically not recognizing the response to some commands until the start of the next one), but that’s also fixed in the latest PyISY. It was dormant for a few years and has recently gotten some much-needed TLC, which is good for anyone using PyISY, including HA. Yay open source!

Good to hear.  I know there's also the branch that @shbatmmaintains pyisy_beta and I contributed to that some z-wave updates.  However I've even tried HACS with his version of PyISY and it's the same error for me.  No idea at the moment.  I started to go through the code but I keep getting side tracked.  I know it's in the nodes/groups.py where it's having an issue though.

I had to backdown several nodeservers I was running because it was too much network traffic to the ISY.  I need to find alternatives for that information I was getting.

[rccoleman]

Just now, simplextech said:

Good to hear.  I know there's also the branch that @shbatmmaintains pyisy_beta and I contributed to that some z-wave updates.  However I've even tried HACS with his version of PyISY and it's the same error for me.  No idea at the moment.  I started to go through the code but I keep getting side tracked.  I know it's in the nodes/groups.py where it's having an issue though.

I had to backdown several nodeservers I was running because it was too much network traffic to the ISY.  I need to find alternatives for that information I was getting.

Yeah, PyISY_beta is what HA is now using as long you install the custom component through HACS, and that has all the new goodness. I do think that the revised custom component and the new PyISY are tied together, so I wouldn’t recommend using one without the other. In any case, I’m glad to see ISY integration get some attention. 

[jec6613]

20 hours ago, rccoleman said:

Yeah, PyISY_beta is what HA is now using as long you install the custom component through HACS, and that has all the new goodness. I do think that the revised custom component and the new PyISY are tied together, so I wouldn’t recommend using one without the other. In any case, I’m glad to see ISY integration get some attention. 

I was using the version through HACS.

[jec6613]

22 hours ago, simplextech said:

It's not a ARM vs x64 issue because it's all Python.  Python doesn't care too much about the underlying system.  However Python is not the most speedy of code in large globs like HASS.

One thing to be aware of is that HASS actually is very heavy in the way it handles components.  A portion of all those 1000+ builtin components gets loaded on startup of HASS regardless of them being used or not.  They are part of the base.  Even though they are small pieces they really add up.

A couple years ago I hit the size limit with HASS which led me out to HomeSeer and other systems because of size and limits.  There's a size limit with the ISY too except they tell you upfront what it is and it's based on the ISY and PLM limitations.  Polisy eventually will help this on the ISY limits (CPU/Memory/Net) but it's still a single system with finite resources the same as any other monolithic system or hub based system.

Yeah, I've noticed that it's quite heavy.  Also that, regardless of me filtering them, it loads them into memory.  I don't care per se about everything being in the UI, just things without the tilde in front of them which is maybe 200 or so items, but it was loading all of them just not displaying them, then crashing out of memory a few hours later.  The really funny thing about how HASS scales up is that it doesn't seem to become CPU or memory or I/O bound, just cruddy coding.  It was running on four cores of a Xeon E5, 8 GB of RAM, and SAS SSD, and never had anywhere near that amount of memory demand.

And this is the problem of open source... really cool features, generally low QC on the code and efficiency.