Help understanding the ISY <--> Hue model?

[andrew67]

Hey guys,  I'm setting up my hue system and have done ok so far, but I am now getting to the point where I need to make some decisions which will take a LOT of ISY programming time.  So I want to try to understand how the hue connection actually works.  Here's a few questions:

1) when the node server was setup, somehow the devices and names were populated in PG3 Hue (sort of) and moved into ISY as nodes.  How/when/how often/etc does this happen?  I will probably be adding/moving lights amongst hue rooms and zones (whats the difference?) and would hate to have to fix ISY code each time.

2) Very little from the hue made it into the ISY, specifically not the names of the lights, only the names of groups.  Any way to fix this?  would make programming easier.

3) The ISY connection involves more than just "triggers."  There seems to be the concept of "levels" associated with ISY and specifically Insteon.  How are these Levels mapped to Hue in the "Command" structure?  i.e. what actually happens when I press paddle up on a scene which includes Hue nodes?  Presumably the "level is carried through" somehow....  Note that this doesn't seem to work quite right.   When I press and hold a paddle, the scene starts out dimmed, but DOES NOT continue to brighten.  Is there a fix for this?

4) what do default and ignore mean in a scene?  I can kind of guess about ignore, but it does not seem to actually work that way...  i.e. If I have a scene and send it a turn off command, it still seems to go off, even if Ignore is selected for the hue.  (This is useful for control of button state on Insteon, which itself is a nightmare).

5) One of the things I would like to do is to create some dynamic scenes.  i.e. make the lights do chases.  What would be the best way to do this?  It appears that setting a light to a color and then turning on "Color loop" works ok.  But there doesn't seem to be a more general way to create color changing effects.  After turning on Color loop, when the lights are switched off and back on, they briefly flash to their previous colors before slowly fading to the commanded color...  What causes/how to fix this?

 

There does not seem to be a Hue reference of any kind other that the VERY helpful one on setup.  Am I missing it?

 

Can someone explain the limitations of the ISY/Hue setup?  xking helpfully pointed out that the hue input devices have fast connections to the hub, but that the hub is only polled by the Node server...  more information like this is extremely helpful, because it tells you what you can't do.... 

 

 

[andrew67]

Partial answers to questions 1 and 2 that I have been able to figure out:

The hue bridge maintains the configuration of hue lights, rooms, groups, and scenes, as well as other items.  This is all documented in the Hue SDK information, which users should NOT have to read to understand how to use ISY and the Node Server.

When setting up your Hue system, you need to remember the following issues/items: 

a) Hue uses Zigbee, which is a multi-hop routed protocol.  There is a "suggested" rate limit which is "recommended"  to be no more than 10 requests per second, with groups perhaps not more than 1 request per second.  No idea what this means in practice, but its definitely SLOW.  Too slow to do any real dynamic effects.

b) You apparently MUST put hue lights into rooms.  This is enforced by the Hue app.  Rooms do not really mean anything more than "group" but you must have each light in one and only one room.  In addition there is the default "all lights" group.  Zones are groups too, but zones can have whatever you want in them.

c) Thus, because you must put each light in a room, its probably better to make the rooms "useful" and to get as much grouping done using rooms, even chopping rooms up into multiple pieces.  There is some personal preference to this, but if you never send a command to a room, then you have wasted an entry, which will end up in your long list of hue stuff that you need to scroll through all the time.  While nothing prevents you from hitting individual lights, which will also be in the list.... read on for more info.

d) The Node server reads the configuration from the Hue Bridge.  You really only want to do this ONCE, and only AFTER you have "named" everything in the Hue App.  This is because the Node server and ISY do NOT actually communicate about the names of nodes.  It is a tortured setup with multiple caches, which will go out of date.

  i) It appears that subsequently added new nodes can be added to ISY by clicking the "Re-Discover Bulbs" button which is visible in ISY when you click on "Hue Bridge."  This only ADDS new nodes (lights, rooms, groups)  to ISY.

  ii) If you remove lights, rooms, zones, etc. they DO NOT disappear from ISY, and cannot be deleted from ISY, you must delete them from the Node server.  You do this by clicking on the nodes "button" in the lower left of the button array for this node server.  This toggles on/off the list of your Hue devices, rooms, and zones.  For each entry, to the right there is a "Delete" button. Deleted nodes do not show up in ISY with status percentages, etc...  Its all blank.  If the name has also been changed in the Hue App, you may need to search the Node server's list for the ADDRESS which is the part to the right of the NODE ID in ISY which is of the form n001_....  and is listed just under the friendly name when you click on the node name under "Hue Bridge" 

  iii) if you have changed the names of lights, rooms, or zones, the Node server WILL get the new name, but the ISY WILL NOT, and there is NO WAY to update.  I suppose you can manually change them in ISY.  The node ID does not change unless you have deleted, and recreated the zone, so the connection still works, even though the name is stale in ISY.

  iv) together, this behavior makes it quite user unfriendly if you don't do it all perfectly first time.

e) sending commands to individual lights works fine.  The upside is you don't need groups, the downside is that ISY uses a "per controller" light command setup, which is painstaking to create.  The alternatives are programs, and using "network resources" but both make the process even MORE obfuscated and complicated.  It is NOT clear how paddle dimming is actually supposed to work, and these choices may interact with that. The commands are not instant, and you will see the popcorn effect.

 

 

[lilyoyo1]

8 minutes ago, andrew67 said:

Partial answers to questions 1 and 2 that I have been able to figure out:

The hue bridge maintains the configuration of hue lights, rooms, groups, and scenes, as well as other items.  This is all documented in the Hue SDK information, which users should NOT have to read to understand how to use ISY and the Node Server.

When setting up your Hue system, you need to remember the following issues/items: 

a) Hue uses Zigbee, which is a multi-hop routed protocol.  There is a "suggested" rate limit which is "recommended"  to be no more than 10 requests per second, with groups perhaps not more than 1 request per second.  No idea what this means in practice, but its definitely SLOW.  Too slow to do any real dynamic effects.

b) You apparently MUST put hue lights into rooms.  This is enforced by the Hue app.  Rooms do not really mean anything more than "group" but you must have each light in one and only one room.  In addition there is the default "all lights" group.  Zones are groups too, but zones can have whatever you want in them.

c) Thus, because you must put each light in a room, its probably better to make the rooms "useful" and to get as much grouping done using rooms, even chopping rooms up into multiple pieces.  There is some personal preference to this, but if you never send a command to a room, then you have wasted an entry, which will end up in your long list of hue stuff that you need to scroll through all the time.  While nothing prevents you from hitting individual lights, which will also be in the list.... read on for more info.

d) The Node server reads the configuration from the Hue Bridge.  You really only want to do this ONCE, and only AFTER you have "named" everything in the Hue App.  This is because the Node server and ISY do NOT actually communicate about the names of nodes.  It is a tortured setup with multiple caches, which will go out of date.

  i) It appears that subsequently added new nodes can be added to ISY by clicking the "Re-Discover Bulbs" button which is visible in ISY when you click on "Hue Bridge."  This only ADDS new nodes (lights, rooms, groups)  to ISY.

  ii) If you remove lights, rooms, zones, etc. they DO NOT disappear from ISY, and cannot be deleted from ISY, you must delete them from the Node server.  You do this by clicking on the nodes "button" in the lower left of the button array for this node server.  This toggles on/off the list of your Hue devices, rooms, and zones.  For each entry, to the right there is a "Delete" button. Deleted nodes do not show up in ISY with status percentages, etc...  Its all blank.  If the name has also been changed in the Hue App, you may need to search the Node server's list for the ADDRESS which is the part to the right of the NODE ID in ISY which is of the form n001_....  and is listed just under the friendly name when you click on the node name under "Hue Bridge" 

  iii) if you have changed the names of lights, rooms, or zones, the Node server WILL get the new name, but the ISY WILL NOT, and there is NO WAY to update.  I suppose you can manually change them in ISY.  The node ID does not change unless you have deleted, and recreated the zone, so the connection still works, even though the name is stale in ISY.

  iv) together, this behavior makes it quite user unfriendly if you don't do it all perfectly first time.

e) sending commands to individual lights works fine.  The upside is you don't need groups, the downside is that ISY uses a "per controller" light command setup, which is painstaking to create.  The alternatives are programs, and using "network resources" but both make the process even MORE obfuscated and complicated.  It is NOT clear how paddle dimming is actually supposed to work, and these choices may interact with that. The commands are not instant, and you will see the popcorn effect.

 

 

 I think the developer assumes someone who is using HUE already knows how hue works and has already set it up prior to trying to use the ISY to control it.

Personally, I use Network resources to control my lights as its quicker and easier than what you have to go through with the Node server (for me). I prefer this method because once the work is done, I dont have to go into the ISY to make changes. I can simply update the scene in the app with whatever changes I want to make. The only thing I miss is being able to dim devices from my switches (which I dont do anyway).

The flexibility the ISY gives to controlling disparate devices also makes building a strong foundation paramount (and time consuming). The Node server currently does not support dynamic scenes as its based on the older API. Until the new api is completed by hue (which allows for more dynamic scenes), I do not see a developer spending the time updating to it only for hue to make changes that break things.

If you actually choose your created hue app scenes, you will avoid the popcorn effect. The ISY is not a zigbee controller so it cannot create scenes within the hue app which is why you get the popcorn effect.

For dimming purposes, the ISY uses standard dimming levels (1-255 I believe. Could be wrong about the exact numbers) for dimming. There can be variances with different devices due to how each protocol/mfg. handles their own dimming. Unfortunately, some concessions had to be made for uniformity purposes so there could be some things off. The fact that the ISY has to talk to the hue bridge, then the bridge has to talk to the devices will create a delay with the dimming commands. As stated earlier, if you want to avoid the popcorn effect, you'll need to control your created hue scene.

The good and bad thing is that the Node server is free. You get what you pay for which means there will be some concessions (though they'd exist even if you did have to pay). Hopefully, the next Node server will answer some of your shortcomings when hue completes the api

 

[Geddy]

15 hours ago, andrew67 said:

iv) together, this behavior makes it quite user unfriendly if you don't do it all perfectly first time.

I think you've hit the nail on the head with this statement. But as @lilyoyo1 points out it's probably best to figure your setup in the Hue app first then add everything to the Node Server to feed to ISY/IoP. 

Another comment (warning) I have is to make sure you put an IP reservation in your router for the Hue Hub. There's a note about it in the readme of the Node Server, but often overlooked. 

"Poly assumes that Bridge IP address never change, so it is recommended that you create an IP address reservation for the Hue Bridge on your router."

I think the biggest thing is it depends what you're trying to achieve with the ISY involved with controlling the Hue lights. Need to plan that out first then it seems fairly simple on/off and levels are simple. Colors get complicated. I like setting up scenes in the hue app then being able to trigger them from the ISY. 

On 9/12/2022 at 5:22 PM, andrew67 said:

Can someone explain the limitations of the ISY/Hue setup? 

This is more or less find out on your own. As @lilyoyo1 points out the current node server is based on the old API so things are limited to a degree. The Hue app allows only so much setup and its own automation that I've found that to work well enough for me not to have it integrated with ISY at all. But that's just how I use Hue in the places I have the bulbs currently. 

[andrew67]

13 hours ago, Geddy said:

This is more or less find out on your own

 

On 9/13/2022 at 6:17 PM, lilyoyo1 said:

I think the developer assumes someone who is using HUE already knows how hue works and has already set it up prior to trying to use the ISY to control it.

 

Guys,  I'm here to let you know that I completely disagree with that kind of thinking.  The whole point of home automation is to simplify our interactions with the places we live, so we can have a better overall experience.  That includes setup.  So we are faced with a choice right now.  We can have the limited automations that Philips, Google, and Amazon can make money on, can have zero support for anything else, or we can hire programmers every time someone wants to change what a wall switch does.  Every time I need to do an update of anything...  I have to re-learn the backward way ISY does things... and become a detective.  It could be better...  It could look for broken links.  It could automatically find problems.  It could have readable logs.  It could have tool tips.  Event viewer could have a decode mode... 

 

[lilyoyo1]

3 hours ago, andrew67 said:

 

 

Guys,  I'm here to let you know that I completely disagree with that kind of thinking.  The whole point of home automation is to simplify our interactions with the places we live, so we can have a better overall experience.  That includes setup.  So we are faced with a choice right now.  We can have the limited automations that Philips, Google, and Amazon can make money on, can have zero support for anything else, or we can hire programmers every time someone wants to change what a wall switch does.  Every time I need to do an update of anything...  I have to re-learn the backward way ISY does things... and become a detective.  It could be better...  It could look for broken links.  It could automatically find problems.  It could have readable logs.  It could have tool tips.  Event viewer could have a decode mode... 

 

You are right. The whole point of automation is to make things simpler. No one ever said it would be easy getting there. Everything that runs well takes putting in work to get there. 

We all wish things were easier but the nature of automation makes it complicated. It's like building a model car.

They have the simple setups for beginners and those who do not want to put much work into it. They can snap a few pieces together and be done. This is basic control systems like hue, Amazon, Google and others like them. They're easy because they're limited. 

Then you have advanced setups. These require thought, skill, patience,  practice, and more. You're responsible for every detail. If you don't plan out what you're doing, you'll have issues. This is automation.

 The very nature of automation is complex. This isn't limited to the isy. Every single full automation controller has it's complexities. This is why C4, Creston, homeworks, and others require expensive classes and dealers to install them. They know it's beyond most people to program and don't want them in the hands of the avg person to screw them up. If a person doesn't want to do the work involved to learn and set things up- this is the path for them. 

UDI could provide detailed information about everything that you ask but then you'd have information overload. Look at the isy cookbook. It's close to 700 pages and is still missing a lot of detail. That's not including the wiki which would add even more. 

Those who take the time to learn about their chosen system generally struggle less with automation systems. Those who jump in without a plan and/or don't take time to learn about what they're using struggle much more. 

UDI can add all that you ask but at what cost? The more they add, the more they'll need to charge for the Isy (controllers themselves). Programming isn't free. They're balancing costs with features and what people are willing to pay. People complain now about polisy choosing 350 bucks. Can you imagine how many people would use UDI if they charged 600? All for something that provides limited return

If a person doesn't take the time (or doesn't want to) to educate themselves, all of that will be for naught.

I agree with you in that the hue Node server could provide more (and better) documentation about setting it up and possibly making changes. As i said before it's free. If it were 50-100 bucks, my expectations would've been much greater. In the end, you get what you pay for. 

Edited September 15, 2022 by lilyoyo1