Why do my programs stop after a day or so whenever I restart my IoX?

[waffles]

(Note: I changed the wait time in the code from originally 25hrs to 29 hrs, as I would miss the heartbeats too often.)

 

I have a bunch of leak sensors that I set up with help of one the posts in this forum.

After a IoX restart the 'monitor' programs seem to launch and run properly (green), but most if not all of them stop (turn red) after a day or so. Usually another manual restart is enough to keep them running for good.  What am I doing wrong here?

This is after a restart:


This is what it often looks like after a day or so:




(Note: I changed the wait time in the code from originally 25hrs to 29 hrs, as I would miss the heartbeats too often.)

[tmorse305]

@waffles, it looks like you have the wait set to 49 hours at least in the one you posted.  Can you show the control 1 and control 2b programs as well?

Are you getting notifications about missing heartbeats?

Edited February 26, 2023 by tmorse305

[waffles]

@tmorse305I initially had the wait times in all of my 2a and 2b programs set to 25 hrs, but then changed a while ago to 49 hrs.

 

here are 1 and 2b:

Do you see anything 'off'?

[oberkc]

Is it possible that your program conditions are such that something triggers the program, but it evaluates as false?  If it were to do so during your 49-hour wait period, the program would halt and run the else path.  The little red line by each program icon means nothing more than the fact that the last program execution was FALSE.

Check the program summary page and see when these programs were last run and status.  

[tmorse305]

3 hours ago, waffles said:

Usually another manual restart is enough to keep them running for good.

@waffles, What are you restarting when you made this comment?  IoX or just running the startup routine again?

The need to change the wait to 49 hours suggests a communication problem with the sensors.  You have a lot of sensors but if they're all in the same area maybe they're having trouble communicating?  Is there a dual band device nearby to handle the transmissions?

My 2a and 2b routines are different than yours.  My triggers do not include the state of the wet sensor.

Geo-VC-2 - [ID 0034][Parent 0032] (this is my 2a)

If
        'Basement / Geo Leak Detector-Heartbeat' is switched On
     Or 'Basement / Geo Leak Detector-Heartbeat' is switched Off
 
Then
        $sLeak_Geo  = 1
        Wait  25 hours
        $sLeak_Geo  = 2
 
Else
   - No Actions - (To add one, press 'Action')
 

Geo-VC-3 - [ID 0035][Parent 0032] (this is my 2b

If
        $sLeak_Geo is 1
    And Time is Last Run Time for 'Geo-VC-2' + 26 hours
 
Then
        Wait  5 seconds
        $sLeak_Geo  = 3
 
Else
   - No Actions - (To add one, press 'Action')
 

 

 

Edited February 26, 2023 by tmorse305

[waffles]

Appreciate your insights @tmorse305 and @oberkc!

 

With 'manual restart' I meant clicking here:

 

 

I have maybe some 100 Insteon devices on the network, pretty much all of the switches, dimmers, etc. are dualband. All that in a fairly typical single family home - nothing too crazy. Overall the system is pretty robust, I get the occasional miscommunication, that I am somewhat familiar with from the home we moved from a few years ago that had a similar setup.

 

Let me post all programs of this leak sensor setup to provide a complete picture

(It's basically an adopted version of this big thread:

 

I originally had started out without IF for the wet sensor, but added it later based on this post:

 

Here it goes:

Control 1, 2a,2b, and 3 is set up for each of the 10 sensors

 

 

These are the 'common' programs:

 

 

 

Last but not least this one is for me to manually reset the variables, if I ever have to do maintenance:

 

 

I have a duplicate set up for the original 25 hrs, but it should be completely disabled:

 

 

I have a few other programs, but nothing (that I am aware of) that should interact with, let alone stop the sensor programs.
 

Do you guys see anything that could cause this stopping problems?

[DennisC]

Not sure if this helps, but when I have a program that runs over 24 hours, I add a run if or then shortly after midnight (12:01 am) within the program.

Edited February 26, 2023 by DennisC
Added clarification

[waffles]

5 minutes ago, DennisC said:

Not sure if this helps, but when I have a program that runs over 24 hours, I add a run if or then shortly after midnight (12:01 am).

Thanks @DennisC; could you elaborate a bit more how you do this?

Also, what would be the benefit for doing this? (Not challenging, just trying to learn here.)

 

Edited February 26, 2023 by waffles

[DennisC]

19 minutes ago, waffles said:

Thanks @DennisC; could you elaborate a bit more how you do this?

Also, what would be the benefit for doing this? (Not challenging, just trying to learn here.)

 

In the past, I have found that sometimes when using run for 24 hours or longer, sometimes the program doesn't keep running past midnight. The exception to this is when using built in scheduling time and the "next day" box. That seem to be ok.

So I add a line that either states "run if" or "run then" pluse program name ànd set the time to 12:01 am.

Which one I use depends on the type of program I am writing. I am traveling and don't have access to my system to post an example.

I hope that helps, if not, try adding it and ask a specific question.

[waffles]

3 hours ago, DennisC said:

In the past, I have found that sometimes when using run for 24 hours or longer, sometimes the program doesn't keep running past midnight. The exception to this is when using built in scheduling time and the "next day" box. That seem to be ok.

So I add a line that either states "run if" or "run then" pluse program name ànd set the time to 12:01 am.

Which one I use depends on the type of program I am writing. I am traveling and don't have access to my system to post an example.

I hope that helps, if not, try adding it and ask a specific question.

I now added this program. I believe it's not quite the same approach as yours, but shouldn't that do the same thing?  (It seemed faster to program):

[DennisC]

Let us know how it works.

[waffles]

Just noticed something:

All 'Leak Sensor- 25hrs' programs show as not enabled (as they should, because I turned them off to not interfere with their 49hrs versions). However the 2a programs show entries under Last Run Time and Last Finish Time. How can this be explained?

[tmorse305]

12 minutes ago, waffles said:

How can this be explained?

 

[waffles]

Hmm, but isn't it disabled, i.e. should not run at all??

 

[DennisC]

Did you stop the running programs before disabling them?

Let us know how your new program works out.

[waffles]

31 minutes ago, DennisC said:

Did you stop the running programs before disabling them?

Let us know how your new program works out.

No, when I set it up, I did not.

However, does this matter? I since rebooted/restarted the IoX several dozen times over the past year or so when introduced my 49 hrs version. I'd expect that after each IoX restart, it would start 'fresh', based on which programs are enabled/disabled. Am I misunderstanding this?

[tmorse305]

Run at Startup may override enable/disable, I'm not sure but it looks like it from your posts.  Easy experiment, reboot with run at start up on/off and see if the disabled 2b programs run.  Honestly I still think you have a communication problem with your sensors.  You shouldn't need a 49 hour wait, the sensors reliably send the heartbeat every 24 hours unless the battery is dying, the heart beat quits before the sensor stops working altogether.  Also on reboot IoX has no idea of the status of the sensor until it receives the first heartbeat.  That might explain why rerunning the startup routine works, because by then there has been a heartbeat.  But if the heartbeats are being missed then it would be hit or miss.

[DennisC]

I didn't go back and read through how you have your programs set up, I did look to see what kind of sensor you are using, but didn't find it. Are these Insteon devices? Are you having any communication issues in the log?

If this was me, I think I would start over. 

If these are battery devices, I would change all of the batteries. 

Then I would re-program the leak sensors using the example in the ISY Cookbook, pages 291 - 297. The Cookbook can be found here:

https://www.universal-devices.com/docs/production/The+ISY994+Home+Automation+Cookbook.pdf

 

[waffles]

So I just did the restart experiment as @tmorse305 suggested.

Looks like disabling the start up program is not effective, as it launched and triggered the 2a's

Since also the 49 hrs 'set' successfully launched, I now have somewhat competing sets of programs....

(In a second round, I also tested the opposite: disabled all 49 hrs, enabled 25. After restart BOTH sets were running again. So it seems the enabling/disabling does not work, at least not how I think how it should work.)

 

My next step is to reinstate only the original 25 hrs set. I'll probably have to delete the other set.

 

The sensors are all Insteon 2852-2222, a mix of v.41, 43 and 44. I had replaced the batteries a few months ago, in an effort to improve the overall signal robustness. I did not notice much change unfortunately, so I came up with 49 hrs flavor.

[waffles]

Update after having run the original 25 hrs set only:

Some of the 2a programs have stopped running again:

Do you guys see anything that could have caused it?

 

 

So above was without @DennisC's suggested daily jump start program. I have activated it now and reset the 'whole thing': 


Let's see how it does now....

[DennisC]

Can you define what you think "stop running" means?

[Geddy]

@waffles Forgive me for chiming in with this, but just for clarification for my own sake and to review what you're doing...

Why do all your programs that appear to update the heartbeat have an "AND" with a status of "Wet" of either on or off? That seems counterproductive. It's as though you combined two programs from the link you provided. They should be mutually exclusive based on reading the early parts of the thread you posted to. 

 

I don't run any Insteon leak sensors (I didn't find them reliable enough at the time I tested them in my system), but looking at your programs compared to the link you originally posted that you attempted to coy from to me you messed yourself up by having an "AND" in the heartbeat 25 hour program. I think that's why you were "missing" heartbeats. Because you were not just basing the program on that instant. 

You've got so many programs and so many edits or variations of them that it's possible something is messing up any changes you make at this point. 

My suggestion would be to put all existing programs in a folder and make the folder conditions false so they don't run.

Start over with the simple programs from the link you provided and try it for a day (or two) and see if you get a correct heartbeat and/or progression of the new programs.

There's something about your initial thought that you are "missing heartbeats" you just randomly changed the program to 49 hours. Heartbeats were hardcoded by Insteon to happen about every 24 hours. If you have a missed heartbeat it's because of other reasons and should be reviewed rather than just kicking the program out 49 hours. (IMO)

 

[waffles]

1 hour ago, DennisC said:

Can you define what you think "stop running" means?

For programs that should be running continuously (for me that's indicated by the GREEN icon), if those switch to a RED Icon. Example here would be the 2a ones that monitor the state of the sensors.


Those also show a "Running Then" in the Summary tab

Am I misunderstanding this?

 

[waffles]

In the meantime the first 2a has already stopped being "green".  In my current understanding #04 is no longer being monitored, so I should get a notification within the next 20-some hours that its heartbeat got missed, correct?

 

[waffles]

hello @Geddy, thanks for chiming it. If I did not want any input, I would not post here

 

The code is from @belias's modified version:
 

top is from the original post - bottom is my implementation
 

 

 

Sorry for the dumb question, but how would I do this "...to put all existing programs in a folder and make the folder conditions false so they don't run..."
(As you may have seen in my earlier posts, I had tried to disable the programs individually by unchecking the "enabled' box, but this did not work.)

Thanks!

 

Edited February 28, 2023 by waffles