Newbie to sys nodes and strings trying to make sense of things

[mango]

Maybe something that checks layer 3 for connectivity every hour, whilst holding the failed send until it can retry with success.

For my cottage (regular power outages - covered with line UPS and generator / occasional DSL internet outages - no cell signal - no 3G/LTE fail over), I was considering something running locally on a Pi or mini PC that the ISY would send to, and give the job of re-try's to a different device.

Edited April 1, 2021 by mango

[MrBill]

8 minutes ago, mango said:

Maybe something that checks layer 3 for connectivity every hour, whilst holding the failed send until it can retry with success.

For my cottage (regular power outages - covered with line UPS and generator / occasional DSL internet outages - no cell signal - no 3G/LTE fail over), I was considering something running locally on a Pi or mini PC that the ISY would send to, and give the job of re-try's to a different device.

I have back up power and failover internet..  I've tried to cover all bases 6 ways to Sunday... the only problem is when the notification times out, nothing retries and there isn't a method to detect failure.  most get thru, but enough don't (1 maybe 2 a month) that I send most notifications by more than one method and that seems to mostly solve the problem, its just ridiculous tho to have to get 3 notifications when the better solution is "If not '200 Ok' then queue for retry" but it's not implemented in the system and there is no way to detect failure at the user programming level.

[Jimbo.Automates]

The NS does set the error status when a message fails to send, it doesn't ignore it, but as of now it has no retry on failure. Will get to that someday.

Also, adding a notification to a program shouldn't be that many steps, create a custom message, restart the NS and add that new node to your program.

Sent from my Pixel 3 XL using Tapatalk

[MrBill]

18 hours ago, Jimbo said:

The NS does set the error status when a message fails to send, it doesn't ignore it, but as of now it has no retry on failure. Will get to that someday

I see the documentation on the error flag.  I had read that before and forgotten it existed because unless I'm missing something it's not that helpful.  Let's use a made up example:  the Mail arrives which is something I get notified for,  I'm at Costco an hour away and won't be home for awhile, the notification fails to send.  OK so I have another program in the ISY that detects the notification fails to send, it sends me a notification that the Notification node server failed to send a notification.... Now what?  I don't have my laptop to investigate logs... I might poke around in a mobile app and determine there no doors open, or water on the basement floor, then check the wireless tags and see if any of them them are in odd states.  There really isn't anyway that I'm aware of to take a Send error and figure which message to try to resend?

18 hours ago, Jimbo said:

Also, adding a notification to a program shouldn't be that many steps, create a custom message, restart the NS and add that new node to your program.

I assume that you a referencing this portion of a sentence that I made above:

On 4/1/2021 at 10:54 AM, MrBill said:

.....but I actually just created 10 new notifications as ISY NR's because It's easier to add one line to a program than the 3 or 4 it takes with the nodeserver.

Unless there are short cut's that I'm unaware of the documentation says that to send one of the "short messages defined by user" to use a then body like this:

Then
        Set 'Notification Controller' Message Good Morning
        Set 'Notification Controller / Service Pushover homeisy' Device JimsPhone
        Set 'Notification Controller / Service Pushover homeisy' Priority Normal
        Set 'Notification Controller / Service Pushover homeisy' Send

Which is my reference to adding 4 program lines above.

To send the ISY Network Resource type of message, the documentation says to use a THEN body that looks like this:

Then
        Set 'Notification Controller / Service Pushover WIND' Priority Normal
        Set 'Notification Controller / Service Pushover WIND' Device JimsPhone
        Resource 'Test.1'

which is my reference to adding 3 program lines to send a message.

[Jimbo.Automates]

On 4/2/2021 at 1:17 PM, MrBill said:

I see the documentation on the error flag.  I had read that before and forgotten it existed because unless I'm missing something it's not that helpful.  Let's use a made up example:  the Mail arrives which is something I get notified for,  I'm at Costco an hour away and won't be home for awhile, the notification fails to send.  OK so I have another program in the ISY that detects the notification fails to send, it sends me a notification that the Notification node server failed to send a notification.... Now what?  I don't have my laptop to investigate logs... I might poke around in a mobile app and determine there no doors open, or water on the basement floor, then check the wireless tags and see if any of them them are in odd states.  There really isn't anyway that I'm aware of to take a Send error and figure which message to try to resend?

I was commented because you had said: 

My biggest problem with all of the above tho is that neither the ISY nor the nodeserver do anything on fail, the message is just lost.

So I was just saying that the nodeserver does do something, you could send a message via another method when you see there was a failure.  yes, you won't know what message failed but something happened... I know it's not ideal right now, but I will try to get to it when my work calms down a little.

On 4/2/2021 at 1:17 PM, MrBill said:

I assume that you a referencing this portion of a sentence that I made above:

Unless there are short cut's that I'm unaware of the documentation says that to send one of the "short messages defined by user" to use a then body like this:

Then
        Set 'Notification Controller' Message Good Morning
        Set 'Notification Controller / Service Pushover homeisy' Device JimsPhone
        Set 'Notification Controller / Service Pushover homeisy' Priority Normal
        Set 'Notification Controller / Service Pushover homeisy' Send

Which is my reference to adding 4 program lines above.

To send the ISY Network Resource type of message, the documentation says to use a THEN body that looks like this:

Then
        Set 'Notification Controller / Service Pushover WIND' Priority Normal
        Set 'Notification Controller / Service Pushover WIND' Device JimsPhone
        Resource 'Test.1'

which is my reference to adding 3 program lines to send a message.

Yes, that's details for custom short messsages, there are also Notify Nodes and I was just commenting that it's not necessary, add the node then set the defaults on the node and trigger the node.  Mostly I'm trying to understand how adding a new NR is easier than using the Notification NS to eventually improve it...

Someday when Chris adds the ability for nodeservers to access custom messages it will get better...

 

[MrBill]

On 4/5/2021 at 1:44 PM, Jimbo said:

So I was just saying that the nodeserver does do something, you could send a message via another method when you see there was a failure.  yes, you won't know what message failed but something happened... I know it's not ideal right now, but I will try to get to it when my work calms down a little.

I currently send almost every notification via 2 or 3 methods, that's how I know how many get dropped.  Lost one just yesterday, the email arrived but the pushover didn't.

Quote

Yes, that's details for custom short messages, there are also Notify Nodes and I was just commenting that it's not necessary, add the node then set the defaults on the node and trigger the node.  Mostly I'm trying to understand how adding a new NR is easier than using the Notification NS to eventually improve it...

I use an excel spreadsheet to make concatenated strings like this one:

token=<redacated>&user=<redacted>&html=1&message=Tag Trouble:\n<b>${sys.node.n004_f693ed5a5dff50.name}:</b> ${sys.node.n004_f693ed5a5dff50.CLITEMP}\nBattery: ${sys.node.n004_f693ed5a5dff50.CV}\n\nTrouble Code: ${var.2.69}\n1-Temp, 2-No Update, 8-Low Battery\n\n[${sys.date} ${sys.time12}]\n

from a data block like this:

Node Name			Node Addr		trbl variable
Extra(usb)			n004_f693ed5a5dff50	69
Living Room Fireplace		n004_1d3698bae66c5e	53
Downstairs Fireplace		n004_b40c6d12d303cc	54
Kitchen refrig			n004_47a90b787eeef5	57
Kitchen Freezer			n004_84e7b435fe8785	59
		

I can speedily make 12 Network Resources and the reason that I choose to do that this time is because Network Resources are simpler to use in programs with just a single program line.   I could do the same type of spreadsheet with nodeserver custom messages, I've just never set up the simplified version.  I also save the spreadsheet so if I ever need the body strings again or need to update node addresses it's fairly simple.

Since you want to understand how to make the nodeserver better, my suggestion would be have a place for the user to set Custom Message default attribute values for  "device",  "priority" and whatever other attributes need a default value, so that they do not have to be called out each time they are used.  For the majority of the notifications I send I never change those value, however priority is bumped in certain cases and it would be relevant to add an additional preface program line to override the default value to 'high'.

On the topic of Notify Nodes and Short Messages, I'm not currently using any of them.  I started out with a lot of Notify nodes for wireless tag failures, but rather than just a verb/action message such as "Low Battery" or "not responding", I kinda like the status report variety of notification, as you can see in my example above, that includes data.  I also had notify nodes for door openings, but ended up deleting them since I couldn't use them in scenes, they are sent only when the $away variable is set, meaning they all needed individual programs anyway.  Notify nodes would work out great for some, but personally I prefer and will stick with Custom Messages.  Same thing with short messages I don't have much use for them, even my mailbox notification contains a variable "Mailbox opened {$var.1.22} times today." so that it's easier to figure out if the mail is being delivered or one of us has already gotten it from the box.

On a new topic, Pushover introduced Custom Sounds with today's app update.  If you made that attribute available in the nodeserver I would definitely make use of it.  From the way I understand it https://api.pushover.net/1/sounds.json?token=(your app token) should return a list of the 21 default sound names, plus whatever sound names the user has uploaded as custom sounds.

So to Summarize all of the above, my suggestions for making the nodeserver better:

1. Retry send on http fail.
2. default attributes for Custom Messages that can be over-ridden on a per use basis.
3. ability to change the sound attribute, including new custom sounds.

 

Thank you for providing the nodeserver, I realize nodeserver developer is your unpaid side job, I've written this message as feedback since you said you're trying to understand how to make the nodeserver better, and those are my thoughts.  I'm also quite certain that I don't always convey my thoughts as thoughtfully as I might like... I  might even come across the wrong way from time to time ?, for that I apologize, it's never my intent to sound demanding and/or complaining and I'm positive that I don't always write with the big picture in mind. ?

 

Edited April 9, 2021 by MrBill

[Jimbo.Automates]

Thanks@MrBill much appreciated, I'm traveling today but will review this in detail when I get a chance. No offense taken, I can be direct in my responses which can come across offensive or defensive when not intended which I don't mean.

Sent from my Pixel 3 XL using Tapatalk

[Jimbo.Automates]

I like your spreadsheet method, very cool.

I'd really like to allow adding those as a custom message string, but Chris is going to allow custom notifications defined in the ISY to be parsed and passed to nodeservers someday so I'm trying to wait for that...

1. Retry send on http fail.

Yes, this has been on the list for a while https://github.com/jimboca/udi-poly-notification/issues/19

2. default attributes for Custom Messages that can be over-ridden on a per use basis.

Will think about this one, let me know your thoughts https://github.com/jimboca/udi-poly-notification/issues/22

3. ability to change the sound attribute, including new custom sounds.

Added: https://github.com/jimboca/udi-poly-notification/issues/21

I can't say when I can get to this because the ELK nodeserver is highest priority right now and I don't have much free time for that, but I will try.

 

[Jimbo.Automates]

Any feedback on those issues?  Also, do you are anyone have a nodeserver log showing what error happens when the message is not sent?  If so, please add to the issue https://github.com/jimboca/udi-poly-notification/issues/19

 

[MrBill]

14 hours ago, Jimbo said:

Any feedback on those issues?  Also, do you are anyone have a nodeserver log showing what error happens when the message is not sent?  If so, please add to the issue https://github.com/jimboca/udi-poly-notification/issues/19

 

The forum never flagged your Monday post as new for me, I just saw it for the first time now and it's below the unread replies line-- weird. 

I don't know what the node server sends because most of mine aren't using it yet, (i still have more network resources than I do node server.) when the ISY fails to send a network resource there is usually a corresponding timeout in the ISY error log.   The problem is it's hard to recreate failed to send or I would do it just to find out.  

I don't know for a fact but it would seem like HTTP 200 OK would be success, anything else could be considered fail.  pushover's API documentation disscusses error responses here: https://pushover.net/api#response  As such it would seem, that 200 is good, anything in the 400's is bad, and of course http timeout is bad.   

I'll work on adding any thoughts to the github issues this afternoon.

[Jimbo.Automates]

Thanks, yes I am looking at their responses, but not all 400 responses should retry. I was testing by creating a bad formatted post just to see that my retry works, but in reality a bad formatted post should not retry.  I was more just curious if you were seeing something other than a 400.

 

[MrBill]

2 minutes ago, Jimbo said:

Thanks, yes I am looking at their responses, but not all 400 responses should retry. I was testing by creating a bad formatted post just to see that my retry works, but in reality a bad formatted post should not retry.  I was more just curious if you were seeing something other than a 400.

 

I think every time I've chased the error down its been http timeoute.  Repeating tho that's always NR/ISY failing...   I think the largest problem is in disappearing Pushovers is internet related... resulting in timeouts because 1)  my crappy rural connection drops packets... 2) pushover hick-ups.. and doesn't respond 3) other internet related issues where a human user would just hit retry.  in other word, lack of response code.

[mango]

I'm back! It's been a hectic few weeks, I am only just able to spend some time on this again.

So, Mr.Bill I have a crude setup working with your strategy of 1 program per NR. but with a weird bug: Example - randomly when a user unlocks, both a lock and unlock status occurs. Do I need to insert a pause, or perhaps a stop command to else to prevent more than one trigger occurring? This happens randomly, not every time the lock is used.

 

Edited April 21, 2021 by mango

[MrBill]

5 minutes ago, mango said:

I'm back! It's been a hectic few weeks, I am only just able to spend some time on this again.

So, Mr.Bill I have a crude setup working with your strategy of 1 program per NR. but with a weird bug: Example - randomly when a user unlocks, both a lock and unlock status occurs. Do I need to insert a pause, or perhaps a stop command to else to prevent more than one trigger occurring? This happens randomly, not every time the lock is used.

<?xml version="1.0" ?><triggers><d2d><trigger><id>31</id><name>[FrontDoor U0] Unlocked Copy Copy</name><parent>5</parent><if><and /><status id="ST" node="ZW002_1" op="IS"><val uom="11" prec="0">0</val></status><and /><status id="USRNUM" node="ZW002_1" op="IS"><val uom="70" prec="0">0</val></status></if><then><net><cmd>6</cmd><parm>12</parm></net></then><else></else><comment></comment></trigger></d2d></triggers>

 

Post your ISY program, click the name in the tree, then right click it and choose copy to clipboard, then paste it into a forum message.

[mango]

HI Bill, I just added a pic. Here is the example code too:

 

[FrontDoor U10] Locked Guest - [ID 0006][Parent 0005]

If
        'Front Door / ZW 002 Front Door Lock' Status is Locked
    And 'Front Door / ZW 002 Front Door Lock' User Number is 10
 
Then
        Resource '[FrontDoor U10] Locked Guest'
 
Else
   - No Actions - (To add one, press 'Action')
 

 

Edited April 21, 2021 by mango

[MrBill]

34 minutes ago, mango said:

HI Bill, I just added a pic. i tried copying and pasting as XML code, but it didn't work. I am remoting in to a local machine to the ISY, clipboard copy may be iffy.

The problem is that both conditions are Status, any status change causes the program IF block to be re-evaluated.  The double notification likely comes when one updates before the other, but not when the other updates before the one.

One way to solve the issue (Disclaimer, someone else might have a better idea.. I'm flying a bit blind because I don't have any z-wave....or locks...) is to use one program to catch the trigger then send it through a series of DISABLED programs to catch the user....

If
        'Front Door / ZW 002 Front Door Lock' Status is Locked
 
Then
        Run Program 'Door User 10' (If)
 
Else
   - No Actions - (To add one, press 'Action')

------------------

[Door User 10] Locked Guest - (DISABLED)

If
     'Front Door / ZW 002 Front Door Lock' User Number is 10
 
Then
        Resource '[FrontDoor U10] Locked Guest'
 
Else  

        Run Program 'Door User 11(If)

-------------------------------

etc.... so the First program is enabled and it catches the lock or unlock... then just daisy chain together a string of disabled programs to figure out which user it was.....

 

If you're not familiar, a disabled program will still run when called by another program, it however won't run itself based on the conditions in the IF statement changing.

[MrBill]

1 hour ago, mango said:

a weird bug: Example - randomly when a user unlocks, both a lock and unlock status occurs.

Thinking about this more, the double trip likely comes anytime the user number is changing.  When the same user that last operated the lock the User isn't changing.  If both change you get the double....

Implementing as I suggested above should resolve the problem.

Edited April 21, 2021 by MrBill

[mango]

Amazing, I understand more now and have a lot of info to work with, thank you.

 

[mango]

I adjusted my programming and naturally everything works perfectly ?

12 users setup, and Pushover is issuing notifications correctly.

Now an easy copy/paste and tweak of the folder for the back door.

[MrBill]

19 hours ago, mango said:

I adjusted my programming and naturally everything works perfectly ?

12 users setup, and Pushover is issuing notifications correctly.

Now an easy copy/paste and tweak of the folder for the back door.

@mangoIf you didn't think of it, I would recommend you create a NR for  "user unknown"  and fire a notification using it in the last Else at the end of the daisy chain of users.  Without it, if for some reason there is not a user match, there is no notification.

[mango]

3 hours ago, MrBill said:

@mangoIf you didn't think of it, I would recommend you create a NR for  "user unknown"  and fire a notification using it in the last Else at the end of the daisy chain of users.  Without it, if for some reason there is not a user match, there is no notification.

Hmm, interesting comment. These particular locks have a user 0 which matches for anything like locking or unlocking the lever manually from the inside or locking from the outside by pressing the Schlage button (locks without having to enter a code). I think that takes care of 'unknown'? I have user 0 setup.

Edited April 22, 2021 by mango

[MrBill]

8 minutes ago, mango said:

Hmm, interesting comment. These particular locks have a user 0 which matches for anything like locking or unlocking the lever manually from the inside or locking from the outside by pressing the Schlage button (locks without having to enter a code). I think that takes care of 'unknown'? I have user 0 setup.

I'd still put something in the ELSE of the last program in the daisy chain....  In theory you will never see that message, but without it you could in theory miss an event if something weird happen to the user.  You could include the ${sys.node.XXXX.YYYY} replacement for the user number just so if you ever do see it, troubleshooting is instantly easier....

The end result of an empty ELSE at the end of the daisy chain of users is that Murphy Law will apply when you desire it least.....

[mango]

Okay understood. I shall do that.

Edit: This has been done for status 'unknown'.

Which segways to another question: The lock has some alarm nodes. I attach a pic, They are currently blank, I am unsure how each reports if they trigger, so ideally I would like to be notified by a push notification also, especially as they are 'alarms'.

This example shows node ZW002_174 Hardware Failure Alarm. Is there a way to find out how this node reports when triggered or is it simply an 'ON' or a '1' ? Is there a trick to interrogate the lock to find out how it communicates?

 

Edited April 22, 2021 by mango