MagicHome Node Server

[markv58]

The double-tap problem seems to be better now with the update but has still persisted on occasion.

Is it possible that these gadgets are kind of going to sleep and need to be nudged to get going again?

Earlier I had to double-tap one but the others behaved after that.

 

[fahrer16]

I'm not sure if they go to sleep if there's no network activity for a certain amount of time. The node server is actively polling the controller's status once a minute by default.  You could try speeding that up by decreasing the longPoll value for the node server in polyglot.  There's not a lot of traffic or processing involved in polling the magichome controllers; unless you have quite a few of them on your network, you could probably be pretty aggressive with that longPoll setting.

[markv58]

@fahrer16 I was still having problems with these little devices so I tested and poked around and have them working flawlessly now. I was having to write some very cumbersome programs to change things but now I can just set a scene.

Modified your magichome.py lines 181, 212, 232, 291, 292, 449 & 450 for now. Have a look.

magichome.modified.py

[fahrer16]

@markv58, I like to define a problem first before jumping to a fix; it's just the engineer in me ?.    What issues were you running into?

It looks like you added quite a few forced queries and updates of the bulbs and changed the behavior of the set color commands to additionally turn on the bulbs. 

The only thing I see that might change the behavior for other users is turning the bulbs on when setting a color.  My thinking when writing the node server was that the color would be set first then it gets turned on so that there wouldn't be a flash of a random color.  I agree it might make sense to have the bulb turn on when setting a color but I'd probably move the command to turn on the bulb afterwards, just in case there's some kind of comms delay and the old color shows briefly.

[larryllix]

27 minutes ago, fahrer16 said:

@markv58, I like to define a problem first before jumping to a fix; it's just the engineer in me ?.    What issues were you running into?

It looks like you added quite a few forced queries and updates of the bulbs and changed the behavior of the set color commands to additionally turn on the bulbs. 

The only thing I see that might change the behavior for other users is turning the bulbs on when setting a color.  My thinking when writing the node server was that the color would be set first then it gets turned on so that there wouldn't be a flash of a random color.  I agree it might make sense to have the bulb turn on when setting a color but I'd probably move the command to turn on the bulb afterwards, just in case there's some kind of comms delay and the old color shows briefly.

Colour flashes were never a problem even for high speed animation that I did. On was always sent after colour and level changes.

For MiLights that was a big problem and the solution was to always dim a light completely, before turning it off. With that system the only way to address a bulb was to send an On first. This could create a flash of the formar colour.
MagicHome / LEDenet doesn't have that addressing sequence problem, so the colour can be set first and then send the On.

[markv58]

The problem is that I have 9 devices, 4 more on the desk waiting and will need probably 4 more to achieve my goal.

I am changing 7 of them right now to set a scene. Putting them in an ISY scene does not work, only 1 or 2 change on the first run and maybe 1 or 2 more after 2 to 4 runs of the scene.

I started changing the scene in a program but the results were about the same until I started doubling the commands. With further testing I started getting 100% results with - Query, SetOn, SetRGBW in the program but that would take 30-40 seconds with the waits that I had to insert in the program to give all the commands time to run.

Setting the delay to 0.1 and changing with a scene using the modifications runs in just a few seconds unless one of the devices falls offline then there is a delay while that gets sorted out but the scene still sets properly.

I believe the problem is with the devices, they aren't $80 Fibaro controllers after all, just little $16 gadgets from China. The control app probably bombards them with packets so they have to respond at some point. The modifications to your program just do that in a more graceful way.

I'm not getting any flashes of light or any noticeable odd behavior from any of the devices as a result of the mods.

This may not be the best way to do it but hopefully it points to a solution.

 

 

[mwester]

There was a thread a long, long time ago that outlined the activities to create a magichome node server on the original Polyglot 1.x -- in that thread I outlined what I'd discovered regarding the message rates that the magichome devices could sustain -- or, to put it the other way around, the minimum delay requires between packets sent to the magichome devices.  Some of the above discussion seems as if those mandatory delays are not being enforced in the code -- you shouldn't ever have to double up commands.

[larryllix]

2 hours ago, mwester said:

There was a thread a long, long time ago that outlined the activities to create a magichome node server on the original Polyglot 1.x -- in that thread I outlined what I'd discovered regarding the message rates that the magichome devices could sustain -- or, to put it the other way around, the minimum delay requires between packets sent to the magichome devices.  Some of the above discussion seems as if those mandatory delays are not being enforced in the code -- you shouldn't ever have to double up commands.

I found if the LEDenet devices were set to "music" mode" off I think they responded to the http packet and if they were in "music mode" they didn't have the slight ramp and didn't respond respond to the http packet. I had no way of telling what the protocol was actually doing and spent almost a year trying to make it work properly. This "music mode" is mentioned in the MagicHome code hacks floating around but no explanation code ever be found.

Now if the controller was ramping the LED levels the CPU in the controller was busy and would not respond until it was done the short level ramp. This ramping delayed response, then caused the RPi driver software to wait until confirmation of the last packet was received. When the controllers were put into "music mode" the controlling packets went out faster, less were dropped, but the LEDs flashed with a distinct On/Off visually harsh "edge".

Most of the time this wasn't visible, but when a sequence of about ten to twenty per second was issued it became very apparent with hiccoughing in the visual sequence. Some animations speeds seemed to work very smoothly and some were nasty and one controller could get ahead of another even though the packets were commanded to be sent out in order.

[Scottmichaelj]

So I was working with the Magichome NodeServer this weekend and I found a couple interesting things.

I am using the Magic Home App on iOS. A few things I noticed was:

-When the LED Controller was ON - with Brightness slider at 0% and the Warm White Slider at 50% the ISY admin console showed the MODE as "COLOR" and Brightness at 50%

Not sure if theres a way to define just when Warm White is on as "White" since the RBG is really connected to the Brightness slider?

-When the color circle on the app was dead center in the color wheel (for white) and changing the Brightness to 50% the ISY admin console showed the MODE as "COLOR" and Brightness at 50% (This seems correct)

-Also when the color circle on the app was dead center in the color wheel (for white) and changing the Brightness to 50% the ISY admin console showed the colors at 1 number lower than the iOS app. I was seeing 128/128/128 for RGB but the ISY was 127/127/127

That said, my needs are simple and this has greatly improved my use of these controllers. So thank you again so much for putting this together!

[fahrer16]

Wow, lots of activity lately!  These devices don't have a single protocol but many different "flavors" with different packet formats and nuances.  I've avoided re-inventing the wheel on dealing with the details of packets being sent to the devices and read back from them.  Instead, my approach is to use the existing flux_led library that has had quite a bit of development.  That way, this polyglot node server is really just an interface between the ISY and that mature library.

1. @markv58 is noting that with his devices, the bulb needs to be queried immediately before a command (possibly to wake it up).  I recall @markv58 had hacked the underlying flux_led code because @markv58's bulbs weren't fully supported by the flux_led library.  I haven't personally seen this issue but the various bulbs of this type do seem to have different behaviors.  Rather than inserting even more queries of the bulb, I'd be interested to know if decreasing the longPoll duration in polyglot helps since that's essentially doing the same thing.
2. @markv58, it sounds like there's a desire to have the bulb turn on when issuing an RGB command.  This wasn't the originally intended functionality but sounds like there wouldn't be an issue so I can implement that easily enough.
3. @larryllix I've noticed the music mode in the protocol and specifically ignored it since I couldn't think of a use case when paired with the ISY and I'm trying to keep it as simple as possible.  Are you saying ramping being done by the controller itself may be causing the appearance of packet drops for @markv58?
4. @Scottmichaelj I don't actually have any bulbs that support warm white or cool white so it's very difficult to test.  Maybe the brightness slider shouldn't be connected to the RGB values when the mode is set to "Warm White"?
5. @Scottmichaelj I think the underlying flux_led project does some rounding when determining the RGB values.  It's not surprising that the values might be off by one integer depending on how the app rounds versus the node server.

 

[Scottmichaelj]

3 hours ago, fahrer16 said:

Wow, lots of activity lately!  These devices don't have a single protocol but many different "flavors" with different packet formats and nuances.  I've avoided re-inventing the wheel on dealing with the details of packets being sent to the devices and read back from them.  Instead, my approach is to use the existing flux_led library that has had quite a bit of development.  That way, this polyglot node server is really just an interface between the ISY and that mature library.

1. @Scottmichaelj I don't actually have any bulbs that support warm white or cool white so it's very difficult to test.  Maybe the brightness slider shouldn't be connected to the RGB values when the mode is set to "Warm White"?

 

I only have the controllers with LED strips myself. The Warm White or Cool White or just really it should be "white" in general. Maybe it would be good to add a "white" mode when its just using the Warm White slider and the brightness is at 0% and show those values in maybe another status box on the ISY called "White" with the slider value percentage? Adding another "brightness" status would be confusing. Probably best to keep the nodeserver close to the actual APP. Thoughts?

[larryllix]

4 hours ago, fahrer16 said:

Wow, lots of activity lately!  These devices don't have a single protocol but many different "flavors" with different packet formats and nuances.  I've avoided re-inventing the wheel on dealing with the details of packets being sent to the devices and read back from them.  Instead, my approach is to use the existing flux_led library that has had quite a bit of development.  That way, this polyglot node server is really just an interface between the ISY and that mature library.

1. <snipped>
2. @larryllix I've noticed the music mode in the protocol and specifically ignored it since I couldn't think of a use case when paired with the ISY and I'm trying to keep it as simple as possible.  Are you saying ramping being done by the controller itself may be causing the appearance of packet drops for @markv58?
3. <snipped>

 

Possibly. The non-music mode slowed things down. the little CPUs in the LEDenet controllers I dealt with did the ramping and AFAICT did not answer the TCP poll until done. 

 

Scott and I never figured out if the LEDenet were the same company or just a cloner of MagicHome protocol. It would be worth a try. In music mode you should see a snappier level change response and possibly a less dropping of packets.

I will try to exemplify this again.
For my tent ceiling animation using ten RGBWW/CW controllers with 5m RBGWW strips, I set set up a round board with ten controllers all controlling a 5m RGBWW LED strip still on the packaging reel. They looked liked old movie tape reels pasted in a pinwheel configuration by my desk computer RPi3.

When writing code to animate say using a simple rotating sequence of some  colour, I noticed at certain speeds some reels of LEDs would actually stay dark and say three or four reels of LEDs later it would suddenly flash for it's proper sequence, but totally out of sequence. I thought the packets on my router were passing each other! :(.
 I spent many months attempting to find out how this could happen. (python http modules  do NOT document any "flush" option and I tried every syntax I could think of).  
Music mode seemed to cure this problem somewhat, but the lights made hard flashes on/of instead of a smoother ramp on/off. The ramps are very quick and hardly noticeable (maybe (0.2-0.3 seconds on or off) except during very fast animations.  When doing a quick light spin on my  test board, I found I could actually detect three or four LED strings on at the same time as they slowly faded off.

[markv58]

7 hours ago, fahrer16 said:

Wow, lots of activity lately!  These devices don't have a single protocol but many different "flavors" with different packet formats and nuances.  I've avoided re-inventing the wheel on dealing with the details of packets being sent to the devices and read back from them.  Instead, my approach is to use the existing flux_led library that has had quite a bit of development.  That way, this polyglot node server is really just an interface between the ISY and that mature library.

1. @markv58 is noting that with his devices, the bulb needs to be queried immediately before a command (possibly to wake it up).  I recall @markv58 had hacked the underlying flux_led code because @markv58's bulbs weren't fully supported by the flux_led library.  I haven't personally seen this issue but the various bulbs of this type do seem to have different behaviors.  Rather than inserting even more queries of the bulb, I'd be interested to know if decreasing the longPoll duration in polyglot helps since that's essentially doing the same thing.
2. @markv58, it sounds like there's a desire to have the bulb turn on when issuing an RGB command.  This wasn't the originally intended functionality but sounds like there wouldn't be an issue so I can implement that easily enough.

 

The hack to flux_led made the bulb discover as RGBW rather than RGB and gave me control of the warm white led's which can be mixed with the RGB led's.

I agree that turning on the device after the RGBW command  would be better however the Magic Light bulb requires the On command first and I've seen no ill effect with the strip controllers.

As to the longPoll or shortPoll, I tested different time spans between commands issued and found that more than 2 seconds would not work.

I have no idea why these things work by issuing the query first but it does. I am not a programmer but I can look at some code and kind of figure out what it does so thats what I did and it's working just fine for me.

[fahrer16]

@markv58, you're definitely seeing some odd behavior with needing to query devices prior to issuing commands.  I wonder if there's something different about the devices you're using or if you have some network issues.  Either way, if that's doing the trick for you I added a new key to the node server (query_before_cmd).  If you add that key and set it to "True" it will query the device immediately before issuing commands.

I also modified the behavior of the "Change RGB" and "Change RGBW" commands so that they will not only change the color but also command the bulb on.

[markv58]

Thank you sir. I will give it a go and see what happens.

As I recall, everything was working just fine and then I added devices 5, 6 &7 and that is when the trouble started. I have a pretty robust wifi network and everything is business class.

I do have a lot of traffic with all the Insteon, Z wave, LEDNet, Lifx, Ecobee and remote sensors, Nest protects, printers, BlueIris, various devices and all.

[Tunaman]

I've got to say that this is certainly proving the value of node servers.

A question:  do you plan on supporting other modes?  For example, I've been experimenting with flux_led and like the gradual mode, but realize that it will take a lot more parameters.  The syntax for the command is flux_led aaa.bbb.ccc.ddd -c gradual speed "r, g, b r, g, b r, g, b"  where the first tuple is the 'fade-from" color, the second tuple is the 'fade-to' color, and the last one the separator color (color it switches to between fade cycles).

 

 

[fahrer16]

I haven’t given much thought to expanding the functionality.  That sounds like it might start to get pretty confusing having so many colors to specify.  

Maybe it would be useful to implement functionality similar to the ‘ramp rate’ specified for insteon devices.  That is, the speed of the shift from the current color to a new color could be specified.

[Charlie_Morlock]

It certainly would make for a crowded screen.

I guess the use case I was thinking about was a visual indicator of weather forecast, fading from low temp to high.

Other than that, I can't think of how useful it would be

Sent from my SAMSUNG-SM-T377A using Tapatalk

[Scottmichaelj]

It certainly would make for a crowded screen.

I guess the use case I was thinking about was a visual indicator of weather forecast, fading from low temp to high.

Other than that, I can't think of how useful it would be

Sent from my SAMSUNG-SM-T377A using Tapatalk

Sports fans could flash every time their team scores.

Slowly ramping bright or dim would be amazing but not sure these controllers are capable of ramping.

[Charlie_Morlock]

They are from flux_led, to a limited extent. You can use the speed parameter to set a percentage for ramp on some functions. There are examples of simulating a sunrise using the time and speed parameters.

The big challenge is determining what all of the functions and associated parameters are, since documentation seems to be limited to what has been reverse-engineered.

Sent from my SAMSUNG-SM-T377A using Tapatalk

[larryllix]

Just a note to anybody involved with writing code for  these Magic Home bulbs. 
Disclaimer: These bulbs are very cheap knock-offs (less than $10 CAD per bulb, shipped from Cali.,  Choifoo mfg.) and may not be the exact Magic Home protocol.

I have found sending a query for status too soon after setting colours, causes the bulbs to freeze their ramping at that point. 
Eg. I send 13 bulbs an On and then a all blue RGB setting command, bulb sequentially from my NRbridge sofware. From another  RPi,  I now send a request for status and I get back 13 different blue colour levels. 
With a following  statReq I get back the same values.  Bulb ramping  appears stalled, giving the StatReq CPU priority over their ramping, I guess.

[sorka]

Can't seem to get MagicHome to discover my wifi LED controller. Using this one with the MagicHome Android app:

 

https://www.amazon.com/gp/product/B01DY56N8U

 

Have the node server installed without error and starting up without error:

 

2019-5-20 23:54:06 - debug: MQTTC: Message: udi/polyglot/ns/3: {"node":"3","start":""}

2019-5-20 23:54:06 - info: MagicHome(3): Processing command: start

2019-5-20 23:54:06 - debug: ISY: 200 - http://192.168.1.52:80/rest/profiles/ns/3/connection/

2019-5-20 23:54:06 - info: MagicHome(3): - NodeServer found on ISY with the name MagicHome. Checking for Nodes...

2019-5-20 23:54:06 - info: MagicHome(3): Completed ReSync with ISY.

2019-5-20 23:54:06 - info: NS: Starting Local NodeServer MagicHome profile number 3

2019-5-20 23:54:07 - debug: NS: Starting log dump of MagicHome log file to frontend client: polyglot_frontend-EKriP

2019-5-20 23:54:22 - debug: NS: Stopping log dump for client: polyglot_frontend-EKriP

2019-5-20 23:54:22 - debug: NS: Starting log dump of polyglot log file to frontend client: polyglot_frontend-EKriP

2019-5-20 23:54:30 - debug: ISY: 200 - http://192.168.1.52:80/rest/profiles/ns/0/connection/

 

Magic Home appears in location 3 in the node server on the ISY:

 

 

The magic home nodeserver can't find any nodes.

 

I tried manually adding a node from ISY NodServers->MagicHome->"Add Node"->"MagicHome RGBW Led" menu where you get the popup where you enter name "node address" and "primary node address".

 

Have no idea what to put. I tried the ip address of the LED controller but it just says "bad request" in red.

 

 

[larryllix]

The controller you show is an LEDenet box and not MagicHome.

 

Further the LEDenet controller shown has 5 channels and not four as I believe most of the MagicHome devices are.

 

These devices take a slightly different protocol to include the 5th channel and the Polyglot node may not be written to self accommodate the change in protocol based on the status request response from the controller.

 

You will need input from @faher16 ??? Here to validate whether his code will support this.

 

Sent using Tapatalk

 

 

 

 

[markv58]

That is the exact same controller that I use around the house, works great.

[markv58]

@sorka On the Polyglot MagicHome configuration page you can input the device parameters and bypass the discovery.