Polyglot Node Server Idea / Request

[TrojanHorse]

Hi, I've been thinking it would be very cool to have a way to grab a "snapshot" of all the current lighting device statuses that could be restored later. 

I make limited use of the ability in 5.x to store status values as variables and recall them later.  I think it would be great to basically be able to scale this up to all the (maybe just lighting) devices. 

My use case would be a way to restore lighting levels after a fire alarm.  I have a program to turn on essentially all of the lights in a fire alarm and leave them on.  On the rare instances of false alarms, it would be nice to have a way to reset the lighting to previous values.  The general use case could be whenever lights are used as some type of notification in the house, they could then be restored to the previous status. 

I have no idea if this is possible or how resource intensive it would be and if polling would be required or if it could just be triggered from changes.  For me, even a "snapshot" that was 5 minutes old would work well enough. 

Curious if this is possible and if anyone else could see a use for this. 

[TrojanHorse]

No one? Actually thinking it would be even better if something like this was native to the ISY.


Sent from my iPhone using Tapatalk

[inovermyhead]

I agree that some kind of “previous overall status restore” function or node server would be useful.  I also have “alarm” programs that elicit attention by turning on lots of lights and flashing them on and off.  When the crisis - or false alarm - is resolved it would be very nice to issue a single command and have everything go back to their pre-crisis state.  Like TrojanHorse I don’t know if this possible without frequent, resource intensive polling

[bpwwer]

Ok, I created a node server that does this.  It still needs a bit of clean up before I publish it.

I have it looking for lighting type devices (basically, devices that have a status with a unit of 0-255 level or 0-100 percent. It will save the current value of the device status and can restore it later.  It currently does all devices that match the above criteria.  

It's not fast.  It takes it a couple of seconds to read the status values on my system and a bit longer to write them all back

[TrojanHorse]

1 hour ago, bpwwer said:

Ok, I created a node server that does this.  It still needs a bit of clean up before I publish it.

I have it looking for lighting type devices (basically, devices that have a status with a unit of 0-255 level or 0-100 percent. It will save the current value of the device status and can restore it later.  It currently does all devices that match the above criteria.  

It's not fast.  It takes it a couple of seconds to read the status values on my system and a bit longer to write them all back

@bpwwer  This is awesome - you are awesome!  I look forward to trying this and integrating it into my system!

[DennisC]

20 hours ago, bpwwer said:

Ok, I created a node server that does this.  It still needs a bit of clean up before I publish it.

I have it looking for lighting type devices (basically, devices that have a status with a unit of 0-255 level or 0-100 percent. It will save the current value of the device status and can restore it later.  It currently does all devices that match the above criteria.  

It's not fast.  It takes it a couple of seconds to read the status values on my system and a bit longer to write them all back

Bob, if you want to test it on another system, let me know.

[bpwwer]

It's ready now. I just need to push it to the node server store.  I'll try to do that today.

It's still a lot slower than I would like.  The main issue is the communication with the ISY.  The call to get the device status (/rest/status) is taking my ISY 10-15 seconds. Some of that is probably because of the number of node servers I have installed,    Restoring is sending individual commands to the ISY for each device it restores so that's dependent on the number of devices and how long it take s the ISY to process each command.

There are really just two controls for the node server: backup and restore.  Both are available for use in programs.

backup takes a snapshot of the current status and restore restores the latest snapshot.  Only one snapshot is saved.  

I wanted to keep this as simple as possible at this point.  

[Michel Kohanim]

@bpwwer,

First of all, you are AWESOME. THANK YOU.

I think the main issue is that /res/status is one at a time. /rest/nodes has everything but it does not have <properties> for node servers. Perhaps that's what we should do for ST so that you will get everything at the same time.

Thanks again.

With kind regards,
Michel

[bpwwer]

It's in the store now, simply called "Backup"

[DennisC]

13 hours ago, bpwwer said:

It's in the store now, simply called "Backup"

Bob,

The install went fine. The backup on my system was relatively quick. The restore took 1 minute 26 seconds to complete. 

One thing worth mentioning is the ISY is locked up during the restore process and doesn't respond to further commands.

[bpwwer]

3 hours ago, DennisC said:

Bob,

The install went fine. The backup on my system was relatively quick. The restore took 1 minute 26 seconds to complete. 

One thing worth mentioning is the ISY is locked up during the restore process and doesn't respond to further commands.

Any idea about how many devices it's restoring?  For reference, I'm backing up/restoring 51 devices.  It takes my system about 22 seconds to restore those.

I've run it a number of times and have not had the ISY lock up.  It's not really doing anything that should cause the ISY problems other than hammering it briefly with a bunch of commands. 

I did find a bug while checking the number of devices.  Every time you run the backup, it adds all the devices instead of replacing the existing ones.  I was to the point where I had run it 4 times and it was restoring each device 4 times (so over 200 commands to the ISY) and the ISY was fine with that.

Possibly there's a device that it's trying to restore that causes problem.  It is guessing as it tries to determine which devices to save and restore by looking at the status UOM. So maybe it's trying to send a DON/DOF to a device that doesn't support that.   Unfortunately, the ISY query only returns the device address and not the other device info like name so it may be hard to determine if one of the devices isn't a lighting type device.

I was a bit afraid that the complexity might snowball on this node server.  If I need to add a way to manually select/deselect devices it starts getting complicated fast.

[DennisC]

Bob,

I ran the backup again, it took 1 minute, 23 seconds this time. The log indicates 108 calls to the IP address. I am also seeing the same address listed more then once in the log.

[TrojanHorse]

I tried setting this up and I'm getting an error.  The error log seems to suggest it's trying port 80.  I'm actually using a different port here.  Is it possible to be able to specify the port? 

[bpwwer]

28 minutes ago, TrojanHorse said:

I tried setting this up and I'm getting an error.  The error log seems to suggest it's trying port 80.  I'm actually using a different port here.  Is it possible to be able to specify the port? 

Yeah, just enter that with the IP address so something like 192.168.1.30:8000 should work.  You might want to wait a bit before moving forward with it though.  There's a bug where each time the backup is run, it duplicates all the devices. It's fixed in my tree but I want to try one more thing before I push out the update.

[bpwwer]

Just missed the store update.  Version 1.0.1 will be in the store in about 1 hour.  It fixes the duplicate bug and switches to use the /rest/nodes endpoint based on @Michel Kohanimcomment about about performance.  The added benefit is that I now get the type and device name so that should make it easier to improve the selection of devices and debug.

[TrojanHorse]

Yeah, just enter that with the IP address so something like 192.168.1.30:8000 should work.  You might want to wait a bit before moving forward with it though.  There's a bug where each time the backup is run, it duplicates all the devices. It's fixed in my tree but I want to try one more thing before I push out the update.

Thanks. I tried that initially. Maybe I have something else that is causing the problem. I will try again with the new version.


Sent from my iPhone using Tapatalk

[bpwwer]

2 hours ago, TrojanHorse said:

Thanks. I tried that initially. Maybe I have something else that is causing the problem. I will try again with the new version.


Sent from my iPhone using Tapatalk

If that's not working still, let me know, I'll probably have to add a config option to add the port.

[DennisC]

First, sorry for delay in responding. 

The nodeserver revision works like a charm. My restore took about 12 seconds. Well done and thank you!

[Screw Loose Dan]

Good job!  I like this a lot and can think of a few use cases.  I just installed and did some testing.  It took ~3 seconds to do the backup and less then 60 seconds to restore (according to ISY Inventory NS, I have 188 nodes.  Looking at the logs from the backup, it appears it attempted to restore 106 nodes).

On 10/16/2020 at 12:10 PM, bpwwer said:

I was a bit afraid that the complexity might snowball on this node server.  If I need to add a way to manually select/deselect devices it starts getting complicated fast

I like the idea of keeping things simple, makes sense.  But, maybe a prefix exclusion, set as a configuration parameter, wouldn't be too hard?  

Also, looks like it's backing up and trying to restore my leak sensors:

$ grep '2E 2 49 2' debug.log 
2020-10-18 15:47:05,999 Controller polyinterface      INFO     backup:discover: Saving 2E 2 49 2/LeakSnsr3-BsmtLaundry-Wet value 0
2020-10-18 15:49:18,955 Controller polyinterface      INFO     backup:restore: Calling http://10.1.10.100/rest/nodes/2E 2 49 2/cmd/DOF

Not sure how easy it would be to exclude, just pointing out a device type there would be no need for a restore.

Another device I would personally like to exclude are the IO Links.  They often seem to have trouble reporting correct values and  in my testing, I changed the value of one with the restore.  This isn't a problem with your code, it's a problem I always have (I generally work around it by programmaticly doing a device query first, which might work in this case as well, I just didn't think about it prior to doing the backup manually).

[bpwwer]

3 hours ago, Screw Loose Dan said:

Good job!  I like this a lot and can think of a few use cases.  I just installed and did some testing.  It took ~3 seconds to do the backup and less then 60 seconds to restore (according to ISY Inventory NS, I have 188 nodes.  Looking at the logs from the backup, it appears it attempted to restore 106 nodes).

I like the idea of keeping things simple, makes sense.  But, maybe a prefix exclusion, set as a configuration parameter, wouldn't be too hard?  

Also, looks like it's backing up and trying to restore my leak sensors:

$ grep '2E 2 49 2' debug.log 
2020-10-18 15:47:05,999 Controller polyinterface      INFO     backup:discover: Saving 2E 2 49 2/LeakSnsr3-BsmtLaundry-Wet value 0
2020-10-18 15:49:18,955 Controller polyinterface      INFO     backup:restore: Calling http://10.1.10.100/rest/nodes/2E 2 49 2/cmd/DOF

Not sure how easy it would be to exclude, just pointing out a device type there would be no need for a restore.

Another device I would personally like to exclude are the IO Links.  They often seem to have trouble reporting correct values and  in my testing, I changed the value of one with the restore.  This isn't a problem with your code, it's a problem I always have (I generally work around it by programmaticly doing a device query first, which might work in this case as well, I just didn't think about it prior to doing the backup manually).

Yeah, I noticed that it was backing up and trying to restore sensor too.  I'm currently looking at devices that have status units of 100 (level 0-255) and 51 (percent) as covers the Insteon switches/lamp type modules (and sensors) and z-wave switch/lamp modules (and possibly others as well).  That's all I had to work with when getting the info via the /rest/status endpoint.

Now that I'm using the /rest/nodes endpoint, not only is is faster, but it also provides some category/type information so I need to refine the filter to look more specifically and dimmer/relay devices.