Home Assistant - Dealing With Insteon Scenes

[vbPhil]

17 minutes ago, ccclapp said:

My post to the HA Github discussion was closed by ../Frenck without addressing the question

github is a tuff place. Populated by Developers for Developers that don't want to hear from customer's/consumer's requests or ideas. Basically, it's a bad idea unless it's theirs. Just my opinion, as I was a developer myself in an earlier life.

[ccclapp]

1 hour ago, Geddy said:

@ccclapp unsure what you want from here either. But this thread was basically dead. It would be best to start a different discussion here. You're taking onto a topic that started almost a year ago and had stalled out a couple of times. 

HI

This may be an old thread, but it is the best discussion of:

1. How isy vs HA handle scenes
2. That isy does and HA does not natively write to devices resulting in: a. eg for a 10 device scene, a single "turn on scene" command, vs 10 separate per device commands AND b. enabeling the scene to function from a controlling e.g. switch, even if the plm/hub is off line.
3. That even though this isy behavior (natively writing to devices) is far superior than how HA handles Insteon scenes, there are issues integrating the isy devices with HA...and some fairly complicated workarounds.

Because of this I reactivated this thread so anyone else considering isy/HA for insteon would benifit from this knowledge.  However, I mistakenly thought by posting these points for the developer of HA to see we might get some current thinking from him. Instead he blocked the question (which admittedly was not posted where it should have been, but was intended to get his eyeballs.

 

In any event, I do have these above related questions and would welcome any insights:

3 hours ago, ccclapp said:

...however, one quick more related question: Am I correct there is decent ISY integration in HA, but not an equally good integration to bringing all HA devices into ISY/eisy if one is considering isy as the primary front end?

 

3 hours ago, ccclapp said:

1. As stated in this thread, isy/IoX writes scene and other data natively to Insteon devices. Also stated, that Zigbee Groups and Z-Wave Associations can be similarly natively written to devices.  My question is: Does ISY/eisy do this with those protocols, or only with Insteon.

2. And: if written natively (by ISY or otherwise) does that help even the playing field between insteon/z-waye/zigbee for lighting (scenes etc)? (not meaning to open a big discussion about pros/cons of each here)

Thanks

 

Thanks!

[Michel Kohanim]

@ccclapp,

With eisy/ZMatter, scenes are written to devices for Z-Wave (called associations) so they will continue to work even without eisy. Very much like INSTEON. Concept of Scenes in Zigbee is a little more complicated.

With kind regards,
Michel

[ccclapp]

32 minutes ago, Michel Kohanim said:

With eisy/ZMatter, scenes are written to devices for Z-Wave (called associations) so they will continue to work even without eisy. Very much like INSTEON. Concept of Scenes in Zigbee is a little more complicated.

With kind regards,
Michel

Thank you Michel. That's great to know. 2 quick follow-ups, if I may:

1. In addition to the benifit of writing to zwave device, does isy-> zwave also send a single Scene/"Association" command, like Insteon, or are commands sent to each zwave device in an association, e.g turn on scene at 60%?
2. Is isy(eisy) unique in this, or do some other hubs/controllers do this too (just asking)

Thanks for taking such good care of us all!

Edited July 18, 2023 by ccclapp

[brians]

How I understand, the concept of Scenes in Insteon is not really the same as how many other platforms do. Insteon was designed from the beginning to not require a centralized controller and has the ability to link Insteon devices together, which they called scenes. Most automation systems today use the idea of central scene, which means everything going through a controller. If one "grew-up" with Inteon and ISY, they would think this idea of centralize scenes requiring a controller as weird, and I can understand the desire to have locally linked devices and not rely on a controller because I have been there myself! This was also back when needed a full computer and hardware to run as a server, which was quite technical and expensive, and not very reliable. Nowadays you can run a whole system on an "inexpensive" Raspberry Pi, or a used PC worth maybe $100 used (which I run mine off - an older Intel NUC). Reliability of these systems is not much of an issue. If you think about how many devices that you rely upon daily which contain computers and software, then seems a bit silly to single out home automation as some special case to worry about. (What is silly is using  devices that require cloud to function).

Home Assistant deals only with devices and entities which are abstracted from actual hardware.  HA doesn't really care what a device or entity is or where it came from, it just knows what type of class/type it is... eg. light, switch, sensor etc.... this makes it easy to do things like easily replace a Zigbee light with a ZWave light, as long as you use the same entity name, and none of the automations need updating. This is also why there is (and probably never will be) no native scenes support between devices.

It would be up to the Addon or Integration to handle this type of native links. This... eg. Zigbee2MQTT or ZWaveJS GUI, before it even sees Home Assistant, and then Home Assistant would not be aware of such links. The issue is that the Insteon integration for Home Assistant does not support this very well . This is why the best way is to run an IoX device using the awesome ISY integration if you are using Insteon. This does however have limitations, eg. you could not make a native scene with a mix of Zwave on HA with Insteon using integration, but there really is no other way. It would be same performance/limitations using just an IoX with Insteon and ZWave/ZMatter.

 

 

 

 

 

Edited July 18, 2023 by brians

[shbatm]

Just want to summarize my understanding of it:

• Insteon Links ≈ Zigbee Bindings ≈ Z-Wave Associations — Device to Device, or Device to Coordinator pairings for communication
• Insteon Scenes ≈ Zigbee Groups ≈ Z-Wave Multicast — Single network commands addressing multiple devices simultaneously
   

Insteon links can be used to control scenes directly from the device (without a modem/coordinator), some Zigbee devices support direct bindings to groups as well.

However, to muddy the water further, each protocol also has 'Scenes' in the sense of 'recallable states and attributes of one or more devices.' For Insteon, this is integral to the second bullet above. Zigbee has partial support for this (Hue & Ikea for example).

Home Assistant only considers scenes in the last sense above, a set of given states and attributes to recall for a given set of devices. These are protocol agnostic, and as I referenced earlier in this thread, the HA developers have been asked on several occasions to consider native support for these external scenes and/or 'one-shot' group protocol commands (by me at least twice), but it is not supportable given the number of overall integrations.

Even for device groups in Home Assistant, it doesn't know how to differentiate between the different protocols. The easiest way I've found is to just create the groups you use frequently in the 'native interface' (for me that's ISY AC, Zigbee2mqtt, and Z-Wave JS) and call those from Home Assistant when you need to or combining those further as Groups in HA when controlling multiple protocols. Yes, it's an annoying workaround, but there's always hope for Matter (/s https://xkcd.com/927)

 

Edited July 19, 2023 by shbatm

[Diesel]

@shbatm

Great job on the integration! Thank you for your hard work.

I just set up the integration between Polisy (IoX) and HA. But the way it handles IoX native Insteon scenes isn't working the way I would expect, and causes major issues with my keypads. 

Here's a scenario to describe the issue; 

3 Insteon only scenes setup in the kitchen, with a keypad button assigned as a controller for each scene. Cooking, Dining & Cleanup. All of these scenes include the same devices (insteon dimmers) for each scene, but the lighting levels are set to different levels.

These scenes used to work perfectly before I added the HA integration, but now when I enable the Cooking scene, both the Dining and Cleanup scene keypads are also enabled.

This shows the same behavior within the HA dashboard. All 3 scenes are imported into HA as switches as expected, but if you turn on the Cooking switch, the Dining and Cleanup switches also get enabled. I see the exact same behavior with other scenes throughout the house that have the same devices as responders.

Am I doing something wrong or is this bug, or expected behavior? 

 

 

 

 

 

Edited July 20, 2023 by Diesel

[shbatm]

1 hour ago, Diesel said:

3 Insteon only scenes setup in the kitchen, with a keypad button assigned as a controller for each scene. Cooking, Dining & Cleanup. All of these scenes include the same devices (insteon dimmers) for each scene, but the lighting levels are set to different levels.

How did you set up the Keypad? Did you use 'group buttons' or set any of them to 'always on' instead of toggle?

What I would do:

• make sure there aren't any 'grouped buttons' on the keypad
• Add the other two buttons to each scene and set the on-level to 'off' (so when Scene 1 turns on, Scene 2 and 3's buttons get an Off
• Bonus 1) if you have another 4th button for 'Kitchen All Off', do the same thing as above to turn off the keypad lights, but you can also set the first 3 buttons to 'Always On' instead of toggle, then they'll always activate their scene when pressed the first time.
• Bonus 2) Make another scene with just the buttons as Responders. Add an ISY Program to check if all lights they control are 'Not On', then: wait 2 sec, turn off the keypad button scene.

[ccclapp]

Hi

@brians and @shbatm Thanks for your clear and thorough replies. That is helpful. I have and will continue to be a loyal and long-term isy user (just got my eisy w zmatter :-).  I am expanding/revamping my overall automation/control setup significantly and will also incorporate HA, which is new to me. As such I am trying to understand the subtilties of protocol options: Insteon, Z-Wave and Zigbee, and the particulars and interactions of multiple hubs (ISY/HA).  You and this string are helping clarify both.  

Here are a few quick follow-ups:

1. A basic question that is still not clear: Can isy insteon scenes be dimmed or their brightness specified (of the scene itself, as I know each device level is set in the scene config) in HA, or via Alexa/Google Assistant, or even within ISY?  I often dim/brighten larger lighting scenes via the KP button, but dont know if and where is can be done virtually.
2. Above @Michel Kohanim confirmed that isy does write natively to z-wave devices in a scene (Associations as @shbatm clarified)  - great! But Michel didnt say if, like with Insteon, for z-wave isy will send a single command to control a zwave scene (Multicast per shbatm). Does ISY do this for zwaye or does it send separate commands for each zw device in a zw scene?
3. If "yes" to the above, getting even more granular: in isy with a scene with BOTH insteon and zw devices, does isy now send separate commands to ALL devices in the scene, or just two, one to each protocol. eg: 1 scene w 4 insteon devices and 5 zwave devices. Is Turn On 1, 2, or 9 commands?
4. For Zigbee, (which is 1 week old on ISY so likely not with 100% capabilities yet) shbatm explains it has similar "direct Binding" and "Group" capabilities, like insteon multi-device links and scenes and zw direct Associations and Multicast. (But Michel also said for Zigbee "scenes" the question was more complicated??). So, all three protocols seem to have similar capabilities in terms of multi-device native links/associations/bindings between same protocol devices, AND single message commands to activate a single protocol scene/group/milticast. Thus my question is: are the three protocols reasonably in the same playing field in these regards (acknowledging differences in communication specifics/speed/distance etc AND assuming the "links" and "scenes" are made by a native controller eg ISY/Zigbee2MGTT/ZWaveJS UI etc, (ie not in HA directly))? Or, are there significant pros/cons of one over another in this regard (meaning, direct "pairings for communication" AND Single network commands addressing multiple devices simultaneously" as @shbatm nicely labels these)? 
5. Does HA have more ability to dim or specify brightness level of "scenes" from external hubs/controllers with ZW or zigbee, than is does with isy insteon? Or can it not specify brightness of externally created "scenes" of any protocol?

Thanks and sorry to dive deeper into the weeds of this topic's question.

 

 

Edited July 21, 2023 by ccclapp

[MrBill]

16 hours ago, Diesel said:

@shbatm

Great job on the integration! Thank you for your hard work.

I just set up the integration between Polisy (IoX) and HA. But the way it handles IoX native Insteon scenes isn't working the way I would expect, and causes major issues with my keypads. 

Here's a scenario to describe the issue; 

3 Insteon only scenes setup in the kitchen, with a keypad button assigned as a controller for each scene. Cooking, Dining & Cleanup. All of these scenes include the same devices (insteon dimmers) for each scene, but the lighting levels are set to different levels.

These scenes used to work perfectly before I added the HA integration, but now when I enable the Cooking scene, both the Dining and Cleanup scene keypads are also enabled.

This shows the same behavior within the HA dashboard. All 3 scenes are imported into HA as switches as expected, but if you turn on the Cooking switch, the Dining and Cleanup switches also get enabled. I see the exact same behavior with other scenes throughout the house that have the same devices as responders.

Am I doing something wrong or is this bug, or expected behavior?

Are you using a single scene, with multiple controllers, where each Controller (red nodes in the admin console) is set differently?  If so, that's not compatible with external control. 

If you are in fact using 3 scenes, where the red node has been configured to set the correct values, then you also should set the root level of the scene the same way.   (note if this is your issue you can prove it by opening the admin console, selecting the root level of the scene and pressing on.  If you got the same result as when the scene was activated via HA then in fact you don't have the root level of the scene configure to turn those buttons off)   

Consider this example:

the button by the front door, is the red node.  The root level of the scene is currently highlighted and named "Front Light - Evening".  When the button by the front door is pushed the settings for the red node is used.  When the ISY itself (or HA by extention) turns the scene on or off the scene settings for the Root level are used instead.

So for this scene the root level and the red node are configured the same.  The responder nodes that end in "bright", "day" and "late night" are in fact other "radio button" style controller buttons at the front door.   Each of the 4 scene's is set to "turn off" the last button. 

 

[Diesel]

17 hours ago, shbatm said:

How did you set up the Keypad? Did you use 'group buttons' or set any of them to 'always on' instead of toggle?

 

Glad to hear this isn't expected behavior. I'm stumped as to what's causing it. 

Haven't changed the secondary keypad buttons from their default. No group buttons, as far as I can tell they're all setup as default toggle. Keypad buttons and scenes were all working fine until I added the HA IoX integration.

For simplicity - I see the exact same behavior with 2 simple scenes in the Den. Watch TV and Relax. I have the opposing keypad buttons added to each scene with the on level as OFF as you described. Yet if I toggle the corresponding 'scene switch' for either scene in HA, the other scene and corresponding button is also turned ON.

I've attached screenshots of both scenes and the keypad settings.

I must have something set incorrectly but I just don't see it.

[Diesel]

3 hours ago, MrBill said:

Are you using a single scene, with multiple controllers, where each Controller (red nodes in the admin console) is set differently?  If so, that's not compatible with external control. 

If you are in fact using 3 scenes, where the red node has been configured to set the correct values, then you also should set the root level of the scene the same way.   (note if this is your issue you can prove it by opening the admin console, selecting the root level of the scene and pressing on.  If you got the same result as when the scene was activated via HA then in fact you don't have the root level of the scene configure to turn those buttons off)   

 

 

Hey, thanks for the help! They're 2 different scenes and a keypad button as controller.

That was my first hunch as well, but everything seems to be set correctly at the root scene and the red node. 

 

But in HA if for example I turn on the switch for Relax, Watch TV is also turned on and vice versa. What am I missing??

Edited July 21, 2023 by Diesel

[MrBill]

@Diesel What happens if you click the root of the scene and then press the on button near the bottom of the right hand pane?  Does it work correctly? or does it behave like turning the scene on from HA does?

[Diesel]

@MrBill

With both scenes currently showing OFF in IoX and HA, turned on the WatchTV scene by pressing the ON button in the root of the WatchTV scene.

This is what the 2 scenes look like now in IoX. In HA both scenes are turned ON. 

 

[MrBill]

@Diesel If I understand your complaint correctly, the undesired behavior is that when send "on" to the scene from Home Assistant all the associated scene control buttons on a keypad turn on. 

If that's the case, what happens to those buttons when the scene is turned on at the root level from the admin console?

The answer to that question will tell us if the scene is set up correctly.  If it behaves the same way as it does with Home Assistant the problem is that the ISY/IoX scene is not working correctly, whether that be from incorrect scene setup or bad links.  If that the case then we continue to troubleshoot on the ISY side.  If on the other hand the Root level of the scene correctly does the job, and Home Assistant is doing something else then we need to take another direction and likely wait for some ideas from @shbatm

[Diesel]

@MrBill

The issue is rather straightforward;

2 separate scenes containing the same Insteon devices just set to different levels for each scene, and a separate secondary keypad button set as controller for each scene.

For simplicity; 

WatchTV set at 80% - Keypad E controller

Relax set at 20% - Keypad F controller

Both scenes and their corresponding keypad buttons behave correctly when the root scene is turned on in Admin Console.

HA integration imports IoX scenes as switches. When the corresponding switch for WatchTV is turned ON in HA, so is the corresponding scene switch to the Relax scene which should be OFF, and vice versa. 

 

[MrBill]

@Diesel I clearly misunderstood the problem. sorry.

 

[Diesel]

1 minute ago, MrBill said:

@Diesel I clearly misunderstood the problem. sorry.

 

No apology my friend. Appreciate you taking the time to try to help. 

The world is a better place because of people like you. Thanks!

[brians]

1 hour ago, Diesel said:

@MrBill

The issue is rather straightforward;

2 separate scenes containing the same Insteon devices just set to different levels for each scene, and a separate secondary keypad button set as controller for each scene.

For simplicity; 

WatchTV set at 80% - Keypad E controller

Relax set at 20% - Keypad F controller

Both scenes and their corresponding keypad buttons behave correctly when the root scene is turned on in Admin Console.

HA integration imports IoX scenes as switches. When the corresponding switch for WatchTV is turned ON in HA, so is the corresponding scene switch to the Relax scene which should be OFF, and vice versa. 

 

 

Insteon scenes don't really have a state. When you press the on or off button in AC, it is just sending the on/off command from the ISY ie. controller to each device in the scene ie. responders. You can press the On using your two different scenes (WatchTV, or Relax) without having to "turn off" a scene.

From my testing, the integration appears to use any of the responders contained in any ISY scene which contains them, to determine a "state" of the switch.scene entity in HA, this is probably so you are able to turn the scene off if want. The issue is when you have multiple scenes containing same responders, turning on/off one switch will turn on/off all the scenes that the switch is a responder to. If used button entities for scene control like on and off (along with other ones like Fast Off etc. ), instead of using a switch entity then this wouldn't happen. The integration does this for other things like queries and beeps so maybe can be done for scenes too.

 

 

 

[Diesel]

1 hour ago, brians said:

 

Insteon scenes don't really have a state. When you press the on or off button in AC, it is just sending the on/off command from the ISY ie. controller to each device in the scene ie. responders. You can press the On using your two different scenes (WatchTV, or Relax) without having to "turn off" a scene.

From my testing, the integration appears to use any of the responders contained in any ISY scene which contains them, to determine a "state" of the switch.scene entity in HA, this is probably so you are able to turn the scene off if want. The issue is when you have multiple scenes containing same responders, turning on/off one switch will turn on/off all the scenes that the switch is a responder to. If used button entities for scene control like on and off (along with other ones like Fast Off etc. ), instead of using a switch entity then this wouldn't happen. The integration does this for other things like queries and beeps so maybe can be done for scenes too.

 

 

 

Ah, so you experience the same behavior? At least it helps me confirm it’s not a misconfiguration issue from my side. Although I’m surprised that @shbatm wouldn’t have mentioned that what I’m experiencing is expected behavior. 

I dont quite grasp what you mean about using button entities instead. Aren’t Insteon keypad buttons just sensor entities in HA without any ability to control underlying devices?

Oddly, before i setup the HA IoX integration I’d been using a Homebridge container with an ISY plugin (Rod Toll?) that also allowed you present native Insteon scenes as switches and behaved correctly in Apple HomeKit. You hit the ‘scene switch’ and it turned on only that specific scene without affecting other scenes that had the same devices (WatchTV and Relax).

I’ve been trying to move to HA to consolidate all my hubs there as automation devices become more and more fragmented. It’s a great platform for this, but this Insteon scene issue is causing some major grief, especially with the wife who uses Homebridge most of the time. 

I’ve thought about using a dummy switch in HA that toggles a variable in IoX that sets a scene state, but that just seems cludgy and cumbersome for all the scenes we use.    

Hopefully we can come up with a better solution that works properly and improve the IoX integration. 

 

 

[brians]

3 hours ago, Diesel said:

I dont quite grasp what you mean about using button entities instead. Aren’t Insteon keypad buttons just sensor entities in HA without any ability to control underlying devices?

 

Home assistant has different entity types. A button entity is one you “press” instead of turning on or off.  Go look at all the entities from the isy integration and you will see some button types in there like query etc. 

[PatPend]

On 7/17/2023 at 7:25 PM, ccclapp said:

Sorry for another follow-up post..

My post to the HA Github discussion was closed by ../Frenck without addressing the question

I therefore made a similar post in the general forum here https://community.home-assistant.io/t/scenes-groups-associations-store-natively-in-device/592910?u=ccclapp if anyone cares

Link not working for me.

 

[ccclapp]

11 minutes ago, PatPend said:

Link not working for me.

 

Yes it was closed by the developer so I reposted in the general forums with a little modification, here.. .

https://community.home-assistant.io/t/scenes-groups-associations-store-natively-in-device-please/592935