Query after change

[ergodic]

Until the Nirvanah days of zero Insteon comm failures comes along, I'm finding it would be useful to have some sort of optional device property "query after updating" or a 0..n retry count with the default=0.

 

The idea is that for devices where the state is important (garage doors, fountains, heart pumps, etc.) or the odd device we all have that never seems to get to 100% reliability, that the ISY will perform an explicit query on the device state anything updates it, and attempt some sort of retry operation after a few seconds in order to fix it if it finds the device state isn't correct. At the very least the ISY is able to correctly record the device state.

 

Right now, I manually code this in programs, but that doesn't help for devices in scenes, and it is also easy to forget.

 

Something a little like this is in Homeseer and it is a useful feature.

[Michel Kohanim]

Hi ergodic,

 

My only concern is that this feature may actually bring down the whole network if more than a few devices have this feature enabled and all those devices are part of the same scene. At the minimum, during the query, no other program can run and the worse case scenario is that your switches may not respond to commands while the query is being done.

 

With kind regards,

Michel

[ergodic]

I suppose I say if you do have more than a few such devices your network is already down - this feature won't do much to change it one way or another. It's precisely for those few troublesome and/or high-importance devices where this is useful.

 

I don't know the internals of course so I'm happy to just talk through my hat: retries can be scheduled randomly and a lowest-priority thread to avoid unreasonable congestion. I'm only talking about a single device query here, not scenes or my-lighting. I can't imagine that hurts that much.

[Michel Kohanim]

Hi ergodic,

 

Let me see if I got it right: selective devices would have flags which simply tells ISY to requery them when the status changes, right?

 

With kind regards,

Michel

[ergodic]

That's it.

 

I don't much care if my guest bath light stays on all night, but I do care if the garage door stays open or the thermostat gets set to zero. For these devices I might even well choose to use the feature as a safety even if my comm reliability for these devices is near 100%.

[ergodic]

I suppose I should also say explicitly that the feature does little good unless the ISY then makes some attempt(s) to rectify an inconsistent state if necessary, as well as just querying the device to verify the status.

[Michel Kohanim]

Hi ergodic,

 

Thanks. How do you suggest ISY rectify an inconsistent state?

 

With kind regards,

Michel

[ergodic]

If the ISY's internal state believes the device should be - for example - on, and it isn't, turn it on. Likewise for an incorrect dim level I suppose.

 

There are secondary semantic questions: Should it do a fast on/fast off? Should it trigger program execution as a status change? What about a device that doesn't respond? I haven't thought all that through, perhaps somebody else has an opinion. As long as it does something to fix it I'm more or less fine with it.

 

It might make sense to separate the action part as a global ISY flag - if a device query (however executed: scheduled program, device requery, etc) uncovers an inconsistent device state then if the setting is enabled it causes the ISY to make some modest attempt to correct the situation. Seems simple and useful.

[Michel Kohanim]

Hi ergodic,

 

I think if we make it simple, there's a much higher likelihood that we can incorporate it into one of the builds. What I am thinking is this:

 

1. A flag on a device which basically says: whenever the state changes on this device, at some point in the future, query it

2. If the query is successful, update the device status

 

With kind regards,

Michel

[ergodic]

Alas I don't see the utility in that implementation: the core idea here is to deal with those frustrating devices in the 90-99% reliability range (which almost everyone seems to have) and whose actual state is 'important' in whatever sense the user deems it.

 

If the ISY is not going to take a corrective action on it's own, and there's no programmatic way to 'scan' out-of-sync devices and fix them, it's hard to see how this helps, but maybe I'm not thinking it through.

[markens]

I've been following this discussion with some interest, as it's a feature I may be able to use to replace programs I have for housekeeping of several "critical" devices.

 

1. A flag on a device which basically says: whenever the state changes on this device, at some point in the future, query it

2. If the query is successful, update the device status

Certainly, keeping it simple will be important for usability. (1) looks good. But what do you mean by update the device status?

 

Let me play devil's advocate for a few thoughts about semantics:

 

An issue that strikes me as important is the notion of which status is correct, the ISY or the device? If the goal is to simply make the ISY and the device in sync with each other, then perhaps it doesn't matter. But what IS important is to define the semantics such that a user knows exactly what the ISY will do in every situation, especially when other devices are involved.

 

There are three distinct situations I'm thinking of:

 

1) The ISY (as controller of a scene, such as via a program) has sent a command to a responder, which did not receive it. The ISY's notion of status is "correct" in this case, and a query will return the "wrong" status.

 

2) A device is controller of a scene, and its command is received by the ISY but not the responder device. Similar to (1), although the ISY did not initiate the state change and therefore has less knowledge of what the intended action was (or correction should be).

 

3) A device is controller of a scene, and its command is received by a responder device but not the ISY. Now the ISY's notion of status is "wrong" and the device query returns "correct" status.

 

Does (3) cause the responder device to report status to the ISY? If not (or if it does, and the update is lost), then the ISY doesn't even know that some action needs to be taken to sync status in this case.

 

So its not clear to me what semantics should apply to a device whenever its status changes in the ISY, since that notion is not really well defined in the context of a "critical" device.

 

Getting back to "simple" semantics...

 

Perhaps it's sufficient for this purpose to define semantics such that it only applies when the ISY commands a responder to do something (case 1). At least in this case, the ISY knows what the resulting state of the device should be. So if a query is done a few seconds later, the ISY can then take corrective action (by resending the command?) if that status is wrong. This could be a very useful feature.

 

Taking the semantic question a bit further: I see this "makeup" command as simply an attempt to correct the device's state. To that end, it would be a "resend" and the ISY should probably not treat it as a separate event in program triggers, etc.

 

Can case (2) be handled as well? If the status changes (and the "housekeeping" flag is set), then perhaps the ISY could proceed with the query/update. But does it do this by resending the same group command it received? I don't know Insteon command semantics well enough to have any idea. And this might have other ramifications.

 

FWIW, here's my situation. The one I really care about is a KPL button linked to an IOLinc relay. The relay responds 95% of the time ok. But for that 5%, it fails and now the KPL led shows the wrong status. (It does flash three times to show the missed ack, but that's easy to overlook.) My ISY does seem to receive the command ok, so I have a program which does a housekeeping query and then updates KPL led appropriately. It seemed better to me in this particular situation to correct the visual status rather than the relay. So this is a variation of case (2) above.

 

--Mark

[ergodic]

If the ISY does a query I'm presuming it gets a valid response; the case where the requery(ies) also fail is more or less hopeless.

 

Case 3 is indeed problematic: there is no way in that case for the ISY to 'know' that it's status is the wrong one when the device is queried. But I'm not sure how important #3 actually is: it's a variant of the how- do- you- know- you're- crazy? problem. Since the ISY didn't hear the state change it doesn't know to issue any verification query at that point in time. At some future time it either issues a controller command to the device or hears one as a responder, and then does it's verify queries at that point, thus devolving to case #1/#2, no? (I also have to say I haven't knowingly experienced any #3s so I don't know how common that case is.)

 

So it seems to me the only way to approach this is to:

 

(A) assume the ISY status is correct and try to fixup the state accordingly,

( assign some sort of 'preferred' state to a device to be assigned when the states are inconsistent and fix it to that if necessary,

© Query the device to update status, but do nothing automatically

 

( might be the best if you look at the actual devices where you'd want to use this feature but I'm fine with (A) too.

 

© by itself seems useless to me and in a conceptual sense also underutilities the capabilities of the ISY to help manage this issue. However if the scripting system had a condition 'if 'device1' is/is not 'inconsistent' you could use © to do whatever you like. For this, the only other hair that has to be added is the device flag that says to check the status or the number of requeries (default=0), or whatever.

 

--Steve

[Michel Kohanim]

Mark and Steve,

 

Thanks so very much for your input.

 

Mark, since ISY does predictive analysis of what the device status should be (in case of scenes), therefore I would venture that ISY's status should be considered as valid in all cases except when ISY, itself, does not "hear" a physical button press. In the "exception" case, there's not much we can do.

 

Now, if we agree with the above premise, then:

1. A device is given a "housekeeping" flag

2. If the housekeeping flag is set AND if the status of the device changes, then:

a. If the status change is a result of a direct button press on a device, then all is good and nothing further needs to be done

b. Otherwise, the device in question is then queried: a) the device does not respond to the query in which case the device is dead/exclamation mark OR the device responds back with its actual status

c. If the status of the device based on the query is step b is the same as what ISY thinks (within 5%), then we are all good

d. Otherwise, ISY should issue the direct command to effectuate the change ... repeat 2a-2d for n times and then give up

 

What do you think?

 

With kind regards,

Michel

[ergodic]

It works fine for me, though I have two thoughts.

 

One: Would this apply in the case of a program issuing an explicit query and the ISY discovering as a result that the device state is 'wrong'? I would think on balance that it should,

 

Two: The software developer in me is somewhat drawn to the programmatic solution -- where the ISY records the inconsistency and then leaves it up to the user what to do about it. ie:

 

if status 'device1' is on wrong

then

send notification to all

set 'device1' on

wait 3 seconds

set 'device1' on

else

set 'device1' off

wait 3 seconds

set 'device1' off

 

"Wrong" here is a general modifier for status that only gets triggered when a query finds the status not correct. (There would probably also be a need for an internal mechanism to avoid the housekeeping queries inside this program lest there be an endless tail recursion of some sort.)

 

The argument for this is flexibility, the argument against of course is that it's a nuisance most of the time. I don't know...

[markens]

Mark, since ISY does predictive analysis of what the device status should be (in case of scenes), therefore I would venture that ISY's status should be considered as valid in all cases except when ISY, itself, does not "hear" a physical button press. In the "exception" case, there's not much we can do.

I have no problem agreeing with this premise. And I think the steps you laid out are sound and address the issue at hand.

 

There should be no problem with extra program triggers based on status since the ISY's status is not changing.

 

One remaining question I can think of: what happens if, after n tries, the ISY is not able to change the device's state? This is really no different from a device failure, I suppose, that the ISY can't do anything about.

 

I'm a big fan of knowing what something will do when it fails, and this notion still nags at me here. It's "easy" to design a system to do tasks. Not so much to design it to fail gracefully and consistently. So the semantics of failure are important. Assuming that the "housekeeping" flag alone enables the syncing behavior automatically, perhaps a program condition which triggers on its failure is all that's needed, so a user can perform whatever other cleanup is desired.

 

There's still the issue of conflicts with other ISY/Instean operation while the cleanup actions are being handled. Especially if housecleaning is enabled on "too many" devices, whatever that number is. Michel, how big a problem do you see this being?

 

One: Would this apply in the case of a program issuing an explicit query and the ISY discovering as a result that the device state is 'wrong'? I would think on balance that it should,

Makes sense to me to do this.

 

Two: The software developer in me is somewhat drawn to the programmatic solution -- where the ISY records the inconsistency and then leaves it up to the user what to do about it. ie:

Hmmm, interesting idea. I've been thinking about this on and off all morning, and I keep going in circles. My gut reaction is that the added flexibility doesn't really add much that can't already be done with existing programming constructs. I've tried to define some specific condition semantics, and it always comes back to too much complexity (all of which must be tested and documented) for too little payoff.

 

The one nugget of this process which seems useful: some concept of a "queryresult" condition which could be tested for specific results (like "can't sync" error above, or "device not responding"). But, again, I'm having difficulty actually defining semantics for it. My brain hurts now...

 

--Mark

[ergodic]

I think the ultimate question is whether this programmatic feature can be folded into the other 'error' condition proposals that have been floating around the forum in various guises. (On comm error, On disabled, whatever).

 

If there's some neat way to collapse all this together then I think it would make sense - otherwise I probably agree on its own that the marginal added flexibility is outweighed by the nuisance coefficient. Mostly I just want the device to do what I said.

[Michel Kohanim]

Hi Mark and ergodic,

 

I actually like the idea of giving the user the control over what should happen. Mark, I am not sure why you don't like this idea.

 

As ergodic suggests, I think we can put all of this into error detection and correction bucket. I am still working on error detection but surely correction either manually or automatically would surely be something very useful.

 

Thanks so very much for your input.

 

With kind regards,

Michel

[markens]

Hi Michel,

 

I actually like the idea of giving the user the control over what should happen. Mark, I am not sure why you don't like this idea.

It's not that I don't like the idea -- user flexibility is good! My reaction was more that the semantics seemed weighty. But I think you're in a much better position to make the call on this after getting input on the functional goals. Thanks much for running with this!

 

--Mark

[Michel Kohanim]

Thank you both for the feedback and input.

 

With kind regards,

Michel