How to Synchronize Program Execution to the Hour

[Jimini]

Thanks, larryllix, for confirming that.  I will make use of that.  

[Jimini]

A new problem just occurred this morning.  My little routine that waits for one hour, increments a state variable and then executes another "Wait 1 hour" failed at the beginning of daylight savings time this morning.  The routine correctly woke up at 1am, but failed to wake up and increment the state variable at 2am or any time after that until I kick started it again.  After the kick start, it has correctly awakened in hour hour intervals.  

Does anyone know if it is documented that the Wait command fails when ISY changes its time twice a year?  

[mwester]

There have been numerous bugs around DST (both times of year).  Some of them are (were) in the ISY firmware, and should be fixed if you're on the latest release.  Many bugs are in user programs that don't know how to deal with the time change... you're on your own for those.  And some of the bugs are in the various devices and computers all over the rest of the world.

I've adopted the brute force but reliable strategy of rebooting EVERYTHING that has an clock, internal or external.  Sounds horrible, but we get power fails rather regularly so things reboot here pretty often, and I've got the startup sequence for the ISY programs pretty much working.

There's certainly value in going through your logs and seeing if you can track it down to a bug in your or UDI's code -- but frankly, DST is one of the world's worst ideas and I personally wish to waste as little of my time and energy on it as I can.

 

[larryllix]

@apostolakisl had a lot of trouble with that. He also uses a lot of 1 hour Waits to keep time clocks.

I think my router hung on the 2 AM time change conflicting with its own self power cycle. I am out of the country and ISP says my modem is still connected. No cloud or network functions working.

Yeah DST is the worst. As a matter of fact why have time zones? So what if everybody goes to work around the same number on the clock ? Think of the computer programming and databases time zones take so we can all use the same numbers. Duh!

Sent using Tapatalk

[Jimini]

I agree that DST is more damaging than helpful. I would prefer 12 months of ST but would take 12 months of either just to get rid of the switch.  For one who has joined meetings both in person and by video for the past 25 years in 3 out of the 4 US time zones, two in Europe and 2 in Asia, I think it might be complicated to eliminate all time zones but having the US on one might be possible.  But before either that or elimination of DST is attempted, it is complete lunacy to have some states or parts of some states not follow everyone else. I can never figure out when to call my cousin in Arizona, whether she is at the same time or not as us.  

But back to my issue about ISY and the time shift, seems like I should ask @apostolakisl.  I'm running firmware version 5.0.15A, which is not the absolute most recent version but close to it.  I've been avoiding upgrading because it's generally a hassle (not just for ISY) unless it's actually fixing something I use.  The problem with this is that I won't know if it fixed this problem for another 6 months.  

I could live with kick starting the programs twice year as long as I happened not to be traveling when that happened.  There are a few programs that I would prefer not to stop while I was away on a trip.  I hope you, larryllix, don't have any "essential" programs hung because your router died.  

[larryllix]

I agree that DST is more damaging than helpful. I would prefer 12 months of ST but would take 12 months of either just to get rid of the switch.  For one who has joined meetings both in person and by video for the past 25 years in 3 out of the 4 US time zones, two in Europe and 2 in Asia, I think it might be complicated to eliminate all time zones but having the US on one might be possible.  But before either that or elimination of DST is attempted, it is complete lunacy to have some states or parts of some states not follow everyone else. I can never figure out when to call my cousin in Arizona, whether she is at the same time or not as us.  

But back to my issue about ISY and the time shift, seems like I should ask @apostolakisl.  I'm running firmware version 5.0.15A, which is not the absolute most recent version but close to it.  I've been avoiding upgrading because it's generally a hassle (not just for ISY) unless it's actually fixing something I use.  The problem with this is that I won't know if it fixed this problem for another 6 months.  

I could live with kick starting the programs twice year as long as I happened not to be traveling when that happened.  There are a few programs that I would prefer not to stop while I was away on a trip.  I hope you, larryllix, don't have any "essential" programs hung because your router died.  

Thanks. The worst part is not knowing.

 

I am confident my stats will do the right thing as they dont depend on anything cloud or LAN to do there jobs.

 

I am 95% confident my ISY will do it's jobs except for my outside lights that are all WiFi.

 

I am thinking my router was set to auto reboot at 3 AM Sunday nights and possibly it powered back up at 2 AM and repeated the loop. I am not sure I will ever find out what happened but I need a better router failure scheme in my ISY.

 

Wikipedia has a pretty good map showing the dozens of countries that have tried DST and gone back.

 

Sent using Tapatalk

 

 

 

[Jimini]

My programs that work off time of day or sunrise/sunset, worked after the DST change without any problem.  It was just the program with the Wait 1 Hour that appears to have stopped.  

This interplay between ISY and LAN/WAN presents interesting "gotchas".  I have a set of programs to monitor the temperature of my "poor-mans" wine closet.  The closet sits under a stairwell but plenty of extra insulation and extra heat mass objects and then a fan that sucks air from the house crawl space.  Here on the central coast, I believe our temperatures are moderate enough to allow holding a fairly stable temperature with help from the cool air in the crawl space. ISY monitors the temperature every hour and records the data in a file in its webspace. But I decided it was a bit awkward to pull that up periodically to look at it.  So I have ISY send an email to my gmail account and then I wrote a Google Script to grab the email message once an hour and  put it in a Google sheet that makes pretty graphs, etc.  But then I found an occasional problem where my ISP takes down our service for a while, usually after midnight, but sometime lasting for several hours. Even though my LAN is working just fine, ISY can't send an email to my Google account operating in the Ether somewhere.  Not much I can do about that, except I added a test to the Google Script, that if it doesn't find an email from ISY when it should be there, it sends me its own message warning me that I'm missing data.  It's also an interesting monitor of how often I'm missing internet service even when I'm not trying to use it. 

 

[larryllix]

Did you know you can record data inside ISY using notifications?

I used it for a few years to troubleshoot my HVAC cycling programs. There is a thread here somewhere started by@MWareman with graphing software that all resides inside ISY.

Sent using Tapatalk

[oberkc]

10 hours ago, Jimini said:

Does anyone know if it is documented that the Wait command fails when ISY changes its time twice a year?

I don't know that the wait "fails", but you should consider the possibility that the time change retriggered you program during the wait period, at which point the program was halted.  I don't know what your program looks like at this point so I can only speculate.

[apostolakisl]

Yes, to the best of my knowledge, the DST clock change kills all waits and repeats.  It used to do that and I believe it still does, though I no longer have programs that run waits/repeats through the night after this issue left my sprinkler on for 24 hours striaght.

If you want to run a program at the top of every hour, this is quite easy now with node server.  Just install ISY time data node and then do:

 

Or, just in case, have it run at any minute that is not 0 just to avoid any confusion that might occur at the zero minute of DST change.

 

[kclenden]

On 2/17/2020 at 8:19 AM, Jimini said:

Question: does that statement '$.Top_of_Hour Init to $.Top_of_Hour' create a trigger for programs waiting for a change of that variable if the variable is a state variable?  I would think not, but just asking to be sure.

Besides what larryllix said, there's another reason "$.Top_of_Hour Init to $.Top_of_Hour" wouldn't trigger programs waiting for a change of that variable.  It's because assigning the same value to a state variable isn't considered changing it and won't act as a trigger.  For example, if $.Top_of_Hour contains the value 10 and sometime later a program executes "$.Top_of_Hour = 10", that won't trigger any programs because the value of $.Top_of_Hour" hasn't actually been changed.

[kclenden]

1 hour ago, apostolakisl said:

Yes, to the best of my knowledge, the DST clock change kills all waits and repeats.

All of my heartbeat checking programs, which use 26 hour waits, were killed this morning.  So my experience agrees with apostolakisl's statement that DST clock kills all waits.

[larryllix]

We have gone through this every year at DST time. UDI stated they will get a fix for it but I don't think there is a fix, only a few workarounds may be possible.

When the clocks move ahead there is an "empty" hour that nothing can happen during that hour because it just didn't exist. When the clocks jump back you have a duplicate hour with the same hour number so a computer hour clock would have 2 for an hour, possibly 3 for a split second and then 2 for another hour and then 3 for another hour?

IMHO any possible solution in the low level ISY engine would likely mess up some ISY programs while others may work OK.

[Jimini]

So many replies, thank you all.  To respond to as many as I can:

I am starting to understand how the Wait 1 Hour might screw up at the clock change based upon larryllix's reply.  That may be the base problem for any waits that cross a clock change.  

My little routine with the Wait 1 Hour, actually increments the state variable, so that part has been working for  a few weeks.  Just died at the clock change.  

It appears to me that apostolakisl's suggestion of using the ISY Time Data Node is the way to go, or at least try.  Would it not be bothered by a clock change?  But I need to know more about this Nodeserve.  I am subscribed to the ISY Portal along the Network add-on, but beyond that, I'm lost as to how to load this ISY Time Data Node.  When I pull down "Noe Servers" from my ISY Console and select Configure, I get an option for Query that seems to do nothing and a long list of numbers, which I take to be nodes, each one marked Empty.  Is this the right place to go for the ISY Time Data Node?  More info would be helpful.  

And one last reply to larryllix suggestion about graphing software inside ISY, I will have to investigate that further.  Interesting to get all that inside ISY instead of sharing the work with Google Scripts. 

Jimini

[larryllix]

So many replies, thank you all.  To respond to as many as I can:
I am starting to understand how the Wait 1 Hour might screw up at the clock change based upon larryllix's reply.  That may be the base problem for any waits that cross a clock change.  
My little routine with the Wait 1 Hour, actually increments the state variable, so that part has been working for  a few weeks.  Just died at the clock change.  
It appears to me that apostolakisl's suggestion of using the ISY Time Data Node is the way to go, or at least try.  Would it not be bothered by a clock change?  But I need to know more about this Nodeserve.  I am subscribed to the ISY Portal along the Network add-on, but beyond that, I'm lost as to how to load this ISY Time Data Node.  When I pull down "Noe Servers" from my ISY Console and select Configure, I get an option for Query that seems to do nothing and a long list of numbers, which I take to be nodes, each one marked Empty.  Is this the right place to go for the ISY Time Data Node?  More info would be helpful.  
And one last reply to larryllix suggestion about graphing software inside ISY, I will have to investigate that further.  Interesting to get all that inside ISY instead of sharing the work with Google Scripts. 
Jimini

I still prefer ISY's internal clock to dependence on another piece of hardware or software and network equipment.

ISY v5 has it's own clock functions available as pulldown menu available system variables.

Another source of real time may still screw up but should recover unlike some ISY wait based infinite loops. Once a wait loop stalls it will not restart again without some special triggers to kick the loop again.

Sent using Tapatalk

[Jimini]

larrylllix, do you mean using the "Time is 1:00:00 After Last Run programxxx" in the IF clause with a Then clause to wake up the program?  Sounds just like a Wait but may use different logic within ISY.  Seems like UD people should be able to recommend this if it solves the problem.  

[larryllix]

27 minutes ago, Jimini said:

larrylllix, do you mean using the "Time is 1:00:00 After Last Run programxxx" in the IF clause with a Then clause to wake up the program?  Sounds just like a Wait but may use different logic within ISY.  Seems like UD people should be able to recommend this if it solves the problem.  

NO. No Waits to be depended on. UDI has made an ISY system clock to be used for Internet synced and self backed up time.

 

 

 

[apostolakisl]

28 minutes ago, Jimini said:

larrylllix, do you mean using the "Time is 1:00:00 After Last Run programxxx" in the IF clause with a Then clause to wake up the program?  Sounds just like a Wait but may use different logic within ISY.  Seems like UD people should be able to recommend this if it solves the problem.  

ISY (v5) lets you access system values for date and time.  However, these values can not be used in the "if" clause.  You must first transfer the system value to a variable.  But, how to transfer the "minutes" to a variable every minute.  You must have a program that runs every minute.  Which puts us back to square one.  DST will tend to mess with programs that are running in wait/repeats running every minute.

If you don't know about polyglot, then you should get on that.  Polyglot vastly expands ISY's functionality.  You either need a raspberry pi or the polisy (sold by UD).  Polyglot runs nodeservers.  Nodeservers currently number about 60 and do all sorts of things like link your ISY to Tesla, Roomba, Husqvarna lawnmowers, Blue Iris Software, several weather apps, and so on and so forth.  You need the nosderver that does time/date.  This lets you directly access all of the pieces of time and date separate from all the other pieces.  Like minutes, hours, day of month, month of year, epoch date, year, month, and so on.  

Polisy is quite a bit more expensive than a rpi.  However, polisy is likely to be the next gen ISY, which will put polisy and isy under the same hood.

[kclenden]

23 hours ago, larryllix said:

IMHO any possible solution in the low level ISY engine would likely mess up some ISY programs while others may work OK.

All depends on how ISY implemented the queue that monitors programs that are waiting.  If they simply set up a table with a pointer to the next line of code to execute and the numbers of seconds to wait (which they would then decrement every second) then it should work just fine through DST change.  If they set up a table with a pointer to the next line of code to execute, and an actual "time to resume", then that would fail during a DST.  Killing all waiting programs would be an ugly way to handle DST changes.  Pretty much all they'd have to do is add an hour to the "time to resume" for all waiting programs during Spring Forward and subtract an hour during Fall Back.  For efficiency sake, if I had been programming the ISY, I would have chosen to save the actual time to resume so that I didn't have a maintenance task that ran every second.

[apostolakisl]

If memory serves me, Michel said that DST is executed by resetting the clock.  Resetting the clock breaks all timed events at the time.  No different than if you manually changed the clock.  Probably a better way to have done it is to run everything on UTC time and let the clock run continuously.  Any time reference within an if clause would reference UTC +/- the locally selected time zone.  Say DST would just change the plus/minus number and not change the clock.  But I could be full of BS.

[larryllix]

34 minutes ago, kclenden said:

All depends on how ISY implemented the queue that monitors programs that are waiting.  If they simply set up a table with a pointer to the next line of code to execute and the numbers of seconds to wait (which they would then decrement every second) then it should work just fine through DST change.  If they set up a table with a pointer to the next line of code to execute, and an actual "time to resume", then that would fail during a DST.  Killing all waiting programs would be an ugly way to handle DST changes.  Pretty much all they'd have to do is add an hour to the "time to resume" for all waiting programs during Spring Forward and subtract an hour during Fall Back.  For efficiency sake, if I had been programming the ISY, I would have chosen to save the actual time to resume so that I didn't have a maintenance task that ran every second.

Yeah, I got thinking about your algorithm to convert all relative times into absolutes and then sort them into a stack that would always only need the next absolute time evaluated. That would present some complexities and each time change would require a  specialised adjustment to be run. Could be done.

Keeping waits as relative times and sorting into a stack, using the same technique, then absolute time would not matter at all. It would be based on a relative clock and no absolute clock would have any effect. Obviously that is not being done in ISY or DST should not be a problem.

I guess there is always a solution. With so many absolute time based events in ISY code, it would seem maybe two timer stacks (one relative (for Waits)  and one absolute time) would be the simplest. Just check both top of stack values every second against it's own clock. No DST special code adjustments should ever be needed.

Reading your post again, I think you just said all that.  

Now there is the problem of 2:30 AM in the spring clock jump, could never trigger because it doesn't even exist......LOL.

Oooops one more. How about 1:30 AM on the fall back which exists twice that night...LOL!!

[kclenden]

On 3/9/2020 at 11:47 PM, apostolakisl said:

Probably a better way to have done it is to run everything on UTC time and let the clock run continuously.

Ding, ding, ding!  That's what I meant to say. 

[larryllix]

Ding, ding, ding!  That's what I meant to say. 

I think with a little thought all Waits could be adjusted twice per year without too much problem but apostolakisls technique may be easier and more foolproof.

Sent using Tapatalk