Proposed Evapotranspiration Algorithm in 3.1.4 Beta

[Mark Sanctuary]

Yeah we have been working out the variables before he released the algorithm. We can move on to the next phase. Go for it Peter and submit the pseudo code.

[peterd]

Will post algorithms later today.

[peterd]

Irrigation Requirement is accumulative and so is Water Deficit. So, at each moment in time, these values reflect how much water has been applied and how much is needed.

Well, unlike Irrigation Requirement, Water Deficit doesn't accumulate past a single 24 hour period.

 

Now, here's the kicker: why do we even need to calculate daily or yesterday's?

I think this is the point I've been trying to make, but we'll see momentarily when I post the algorithms.

The only difficulty would be to figure out what we mean by Rain "Today".

Well, I think it's important to keep in mind that while WB imposes a definition (and behavior) on their Rain Today value (accumulative throughout the day, then reset after midnight), we don't have to treat our rain tally the same way.

 

Peter, we can add that command (just like reset and cycle complete) but, as Mark said, I think it's just going to cause confusion.

I'm not trying to force-fit anything, but I actually think there's a certain elegance to the approach. Let's look at the algorithms, and if you can please post proposed syntax for these 3 commands, we can work up some sample irrigation programs to see how straightforward (or not) it looks.

 

Peter

[peterd]

OK, here goes.

 

Note: All variables dealing directly with water are in units of depth (mm internally, entered/presented as mm if Metric presentation chosen in Climate config or as inches if English chosen)

 

Variables

In the Climate Module configuration:

// The amount of water the user's irrigation program applies to the soil during each run.
Water_Applied_per_Irrigation = (Value entered by user)

// To adjust for the differing ET in inland vs coastal areas.
Region_Factor = (Value based on location selection by user)

// To adjust for the relative efficiency of soil absorption
// Used by ISY only for absorption of rain - the user is responsible for factoring this into the irrigation run-times in their program.
Soil_Absorption_Factor = (Percentage selected by user)

Internal to ISY:

// A single instantaneous ET calculation result
Evapotranspiration_Sample

// The sum of all Evapotranspiration Samples since the last Daily calculations
Evapotranspiration_Tally

// The number of Evapotranspiration Samples since the last Daily calculations
Sample_Count

// Allow WB's "Rain Today" (a midnight-to-midnight total) to be used for Daily calculations whenever the user's irrigation program requests
Total_Rain
Previous_Rain_Total

Available to users' irrigation programs:

// The average of all Evapotranspiration Samples since the last Daily calculations
Daily_Evapotranspiration

// The Daily_Evapotranspiration less whatever rain has been absorbed by the soil since the last Daily calculations
// This is a signed float and can be a negative value in the event of lots of rain
Daily_Water_Deficit

// A running balance (can span multiple days) of Daily_Water_Deficit since the last time the user's program triggered a decrement or reset.
Irrigation_Requirement

 

 

Algorithms

At each WeatherBug update:

Evapotranspiration_Sample = // EXISTING INSTANTANEOUS ET CALCULATION HERE

// Keep a running total (not a running average) of all samples for a 24 hour period
Evapotranspiration_Tally = Evapotranspiration_Tally + Evapotranspiration_Sample
Sample_Count = Sample_Count + 1

Daily at 23:59:59:

Previous_Rain_Total = Rain_Today // Rain_Today is the existing WeatherBug variable

Daily, when the user's irrigation program issues the "Perform Irrigation Calculations" command:

// Assign new value to Total_Rain based on whatever rain occurred between previous daily calculations and 00:00 plus Rain_Today since 00:00.
Total_Rain = (Previous_Rain_Total - Total_Rain) + Rain_Today

// Calculate average for past 24 hours
Daily_Evapotranspiration = Evapotranspiration_Tally / Sample_Count
// ...and reset tally
Sample_Count = 0
Evapotranspiration_Tally = 0

// How much water left the soil in past 24 hours? (may be negative if more rain than evapotranspiration occurred)
Daily_Water_Deficit = Daily_Evapotranspiration - (Total_Rain * (Soil_Absorption_Factor / 100))

// What is the running balance of water to replace through irrigation?
Irrigation_Requirement = Irrigation_Requirement + Daily_Water_Deficit
// May NOT be negative (soil can't be > 100% full of water)
IF Irrigation_Requirement < 0 THEN Irrigation_Requirement = 0

 

The user's irrigation programs:

// Program 'Irrigation Scheduler' (Enabled)
If
       Time is 11:00:00PM

Then
       Trigger 'Perform Irrigation Calculations'
       Run Program 'Irrigate If Needed' (If)

Else
  - No Actions - (To add one, press 'Action')


// Program 'Irrigate If Needed' (Not Enabled)
// The IF condition decides whether to water by comparing the Irrigation Requirement balance to a constant
// The user is responsible for setting this constant equal to Water Applied per Irrigation
// (and updating it if they change the value of Water Applied per Irrigation in their configuration)
If
       Module 'Climate' Irrigation Requirement >= 0.5

Then
       Set 'Front Yard' On
       Wait  10 minutes
       Set 'Front Yard' Off
       Wait  3 seconds
       Set 'Back Yard' On
       Wait  5 minutes
       Set 'Back Yard' Off

       Trigger 'Reduce Irrigation Requirement'

Else
  - No Actions - (To add one, press 'Action')

// The final THEN action (after the irrigation zones all run) triggers ISY internal calculation of:
// Irrigation_Requirement = Irrigation_Requirement - Water_Applied_per_Irrigation

Please understand that in the absence of any proposed syntax for the system commands within programs I have arbitrarily written those lines of ISY code using a fictitious keyword "Trigger." I will happily edit my post as soon as Michel lets us know what the syntax of these commands should be.

[Mark Sanctuary]

I like it! It's a print!

 

I just want to understand this further...

 

Previous_Rain_Total = Rain_Today // at 23:59:59

 

Total_Rain = (Previous_Rain_Total - Total_Rain) + Rain_Today

 

2.0 total rain at irr trigger Friday = (3.0 End of day total timer triggered Thursday at 23:59:59 - 2.0 last total rain at irr trigger Thursday) + 1.0 current rain measurement at irr trigger Friday

 

Does this make the stationary Rain_Today variable usable at any time of the day?

[peterd]

I just want to understand this further...

Previous_Rain_Total = Rain_Today // at 23:59:59

Total_Rain = (Previous_Rain_Total - Total_Rain) + Rain_Today

 

2.0 total rain at irr trigger Friday = (3.0 timer triggered Friday at 23:59:59 - 2.0 last total rain at irr trigger Thursday) + 1.0 current rain measurement Friday

You've got it!

Does this make the stationary Rain_Today variable usable at any time of the day?

Exactly!

[Michel Kohanim]

Peter,

 

Thank you.

 

What I do not understand is this:

--Daily at 23:59:59:--
Previous_Rain_Total = Rain_Today // Rain_Today is the existing WeatherBug variable

--Daily, when the user's irrigation program issues the "Perform Irrigation Calculations" command:--
// Assign new value to Total_Rain based on whatever rain occurred between previous daily calculations and 00:00 plus Rain_Today since 00:00.
Total_Rain = (Previous_Rain_Total - Total_Rain) + Rain_Today

 

So, let's plugin some numbers:

Previous _Rain_Total = 2

Rain_Today = 3

Total_Rain = 7

 

The user runs the irrigation calculation program:

Total_Rain = (2 - 7) + 3 = -2

 

The user runs the irrigation calculation again within a few minutes:

Total_Rain = (2 - (-2)) + 3 = 7

 

In effect, if anyone runs that program by mistake more than once - and before Rain_Today is moved to Previous_Rain_Total - there will be a toggling effect between two values. In this case -2 and 7.

 

I do not think this is going to work.

 

With kind regards,

Michel

[peterd]

OOPS!

 

Back to what I said about variable names, renaming these helped me see how to fix it. (To be honest, it's quite late and I'm not 100% positive this is right but I think so. Please check this & see.)

 

// At 23:59:59
Post_Previous_Calc_Partial = Rain Today - Pre_Calc_Partial

// When "Perform Irrigation Calculations" command runs
Pre_Calc_Partial = Rain Today
Total_Rain = Pre_Calc_Partial + Post_Previous_Calc_Partial

 

 

You do raise an interesting point, though, about the consequences of irrigation calculations being performed multiple times within 24 hours. My suggestion is to prevent this within the code. I've updated this below. I've incorporated a bit of leeway (22 hours instead of 24) for times when the user might adjust the schedule a bit. I don't think running the calculations early by a bit (< 10%) will throw things off enough to matter.

 

Daily, when the user's irrigation program issues the "Perform Irrigation Calculations" command:

// Do not allow the values to be updated if the last run hasn't been at least 22 hour ago
IF Time_Now - Time_of_Prev_Calc < 22:00:00
THEN return // no update at this time
ELSE Time_of_Prev_Calc = Time_Now


// Assign new value to Total_Rain based on whatever rain occurred between previous daily calculations and 00:00 plus Rain_Today since 00:00.
Pre_Calc_Partial = Rain Today
Total_Rain = Pre_Calc_Partial + Post_Previous_Calc_Partial

// Calculate average for past 24 hours
Daily_Evapotranspiration = Evapotranspiration_Tally / Sample_Count
// ...and reset tally
Sample_Count = 0
Evapotranspiration_Tally = 0

// How much water left the soil in past 24 hours? (may be negative if more rain than evapotranspiration occurred)
Daily_Water_Deficit = Daily_Evapotranspiration - (Total_Rain * (Soil_Absorption_Factor / 100))

// What is the running balance of water to replace through irrigation?
Irrigation_Requirement = Irrigation_Requirement + Daily_Water_Deficit
// May NOT be negative (soil can't be > 100% full of water)
IF Irrigation_Requirement < 0 THEN Irrigation_Requirement = 0

[Mark Sanctuary]

I do not think this is going to work... without some safeties put in place.

With this being pseudo code I guess I assumed that it would be ran every 24 hours. Peter is right it will work as expected if we just prevent execution of this part of the code too often, about every 23 hours or so. We are going to have gotchas but with safeties in place it should work great. The function of this code is looking very good and I think it is about ready for a round of alpha work. Plus after it is implemented and being tested you can put examples here of, "Guys I found this issue when I execute it, what do you think?" and we can help.

 

Good job Peter!

[peterd]

[...] it will work as expected if we just prevent execution of this part of the code too often, about every 23 hours or so.

Yeah, I went with 22 since 2 hours amounts to ~8%. I figured anything under 8% isn't going to break anything...

 

Good job Peter!

Well, we'll see what Michel thinks. You know I couldn't have done it without you & Michel. (But then, we couldn't do any of this without Michel - he's the one to thank!)

 

Once the beta comes out, my real work begins. I mentioned redoing my irrigation setup. I now (with help from some gardening professionals - my thumbs aren't green enough alone) have 20 zones to program! (Double Facepalm)

[Mark Sanctuary]

Peter,

 

Do you want to help with the Wiki on this topic?

 

If so, have you gotten UDI Wiki access yet?

[peterd]

Peter,

 

Do you want to help with the Wiki on this topic?

Yes. We can start with the ET overview which was the 1st of my most recent "Big 3" posts. Then I figure once we have the beta in hand we can implement initial programs & write that up. I'd also like to get IndyMike & whomever else back east involved in testing & reporting experiences since we only represent a couple of climates ourselves.

 

If so, have you gotten UDI Wiki access yet?

If you mean to post/edit then not yet.

[Mark Sanctuary]

Great!

 

I have started with just the basic link structure in the how to guide section.

 

How-To_Guide - Irrigation

[Michel Kohanim]

Mark,

 

Please REMOVE ALL the Wiki contents you posted IMMEDIATELY. It's one thing to have discussions about different methods on a public forum and yet antoher to post those ruminations and speculations on a FACT based WiKi. This will add a lot of confusion. Furthermore, we do not consider blah blah as words that can be used in our WiKi. Thank you.

 

Excellent dicussions but I think we are moving farther and farther away from a simple solution and elegant soultion:

1. Now we have to keep partial counts of rain amount just becuase we do not believe today should actually be today. Today is actually 1/3 yesterday and 2/3 today!

2. We have to add spagetti code (not common to anything else we do in Programs) to prevent the users from calling the calculate program more than once within some (hardcoded) period

3. We have to explain all this to those users who just want a simple value: should I water the plants today or NOT?

 

I will have to think about the actual solution but it shall leave complexities to be solved by advanced users using programs and variables.

 

Thanks and with kind regards,

Michel

[Mark Sanctuary]

Please REMOVE ALL the Wiki contents you posted IMMEDIATELY. It's one thing to have discussions about different methods on a public forum and yet antoher to post those ruminations and speculations on a FACT based WiKi. This will add a lot of confusion. Furthermore, we do not consider blah blah as words that can be used in our WiKi. Thank you.

No worries Michel. I will remove the links for now. Sorry about the Blaa blaa but really its only a placeholder that I was going to replace later today, at which point I was going to be pasting actual text into it. Really don't see any reason why we can't work on the explanation of Evapotranspiration since it is already a part of the tool in 3.1.3 release. As for any code examples they will not be there until the tool supports the commands.

[Mark Sanctuary]

Excellent dicussions but I think we are moving farther and farther away from a simple solution and elegant soultion:

1. Now we have to keep partial counts of rain amount just becuase we do not believe today should actually be today. Today is actually 1/3 yesterday and 2/3 today!

2. We have to add spagetti code (not common to anything else we do in Programs) to prevent the users from calling the calculate program more than once within some (hardcoded) period

3. We have to explain all this to those users who just want a simple value: should I water the plants today or NOT?

Thank you for being open about what your thinking. I makes it much easier for us to help you and understand the pitfalls your seeing that we may not understand from an outside perspective.

 

Yes spaghetti code it is... because I can't think of any other ISY program commands that are limited to a window of time they can function as this trigger command idea. If the command trigger 'Perform Irrigation Calculations' part is that concerning I think we all agree you can just not do that part and sync it with the WeatherBug. In the end we just want to make the Evapotranspiration data useful.

 

This overall design is a good idea for what we are trying to do here with splitting some functionality with the ISY side and some functionality with the user programs side, the only thing easier on the user really would be a full blown irrigation module but that option is not on the plate.

 

Is there any other issues outside of this rain/spaghetti/trigger code issue in this design that we should discuss that is holding up it being implemented?

 

Thanks,

[Michel Kohanim]

Hi all,

 

Even though what I am going to propose is going to add more events and calculations to ISY but I think it will have the following benefits:

1. Alleviate the need for redefining what is "today"

2. Gives the users to ability to use one variable (Total Water Deficit) to decide whether or not they need to water the plants

 

Here's the high level design:

1. We will keep running daily Irrigation Requirements for the past 7 days (including today)

2. Irrigation requirements are calculated at 00:00 every day

3. Total Water Deficit is the total water needed after applying water and/or rain

 

This way, one may always use Total Water Deficit and be relatively certain that the amount of water applied is optimum. For those who have advanced requirements, they can use Yesterday's values, today's values, 3 days ago, etc.

 

With kind regards,

Michel

[peterd]

Michel -

 

It sounds like you are reversing the meaning of "Water Deficit" and "Irrigation Requirement." If that's the case, I would caution against doing that since "Irrigation Requirement" has a specific meaning in the Irrigation Requirement Balance model of ET-based irrigation which we're implementing. Thus, it won't be possible to say "if you have questions, read such-and-such article."

 

Peter

[peterd]

Michel -

 

Until floating point variables are implemented, we could certainly keep it even simpler (this is a 1st implementation, after all). I honestly haven't been able to see how Daily Evapotranspiration or Daily Water Deficit would be used in anyone's programs. If you want a simple first implementation, then I think all you'd have to surface to programs would be Irrigation Requirement (the running balance).

 

I would be happy to post algorithms for a simple approach with calculations @ 00:00 and no confusion about Today.

 

Peter

[Michel Kohanim]

Hi Peter,

 

Apologies for confusing the variable names.

 

Yes, if you would be so kind as to start us off with the first pass at implementation algorithm. Once we have the first pass implemented, I am sure we will get a LOT more feedback and we can decide what's best for everyone.

 

Thanks so very much and with kind regards,

Michel

[peterd]

Michel -

Respectfully, please consider the following proposal for a simple implementation with no ambiguity about "Today", etc.

 

Variables

In the Climate Module configuration:

// The amount of water the user's irrigation program applies to the soil during each run.
Water_Applied_per_Irrigation = (Value entered by user)

// To adjust for the differing ET in inland vs coastal areas.
Region_Factor = (Value based on location selection by user)

// To adjust for the relative efficiency of soil absorption
// Used by ISY only for absorption of rain - the user is responsible for factoring this into the irrigation run-times in their program.
Soil_Absorption_Factor = (Percentage selected by user)

Internal to ISY:

// A single instantaneous ET calculation result
Evapotranspiration_Sample

// The sum of all Evapotranspiration Samples since midnight
Evapotranspiration_Tally

// The number of Evapotranspiration Samples since midnight
Sample_Count

// The average of all Evapotranspiration Samples for the last full day
Evapotranspiration_Yesterday

// Evapotranspiration_Yesterday less whatever rain was absorbed by the soil yesterday
// This is a signed float and can be a negative value in the event of lots of rain
Water_Deficit_Yesterday

// A running balance (can span multiple days) of Water Deficit since the last time the user's program triggered a decrement or reset.
Irrigation_Requirement

 

 

Algorithms

At each WeatherBug update:

Evapotranspiration_Sample = // EXISTING INSTANTANEOUS ET CALCULATION HERE

// Keep a running total (not a running average) of all samples for the day
Evapotranspiration_Tally = Evapotranspiration_Tally + Evapotranspiration_Sample
Sample_Count = Sample_Count + 1

Daily at 23:59:59:

Rain_Yesterday = Rain_Today // Rain_Today is the existing WeatherBug variable

// Calculate average for past 24 hours
Evapotranspiration_Yesterday = Evapotranspiration_Tally / Sample_Count
// ...and reset tally
Sample_Count = 0
Evapotranspiration_Tally = 0

// How much water left the soil in past 24 hours? (may be negative if more rain than evapotranspiration occurred)
Water_Deficit_Yesterday = Evapotranspiration_Yesterday - (Rain_Yesterday * (Soil_Absorption_Factor / 100))

// What is the running balance of water to replace through irrigation?
Irrigation_Requirement = Irrigation_Requirement + Water_Deficit_Yesterday
// May NOT be negative (soil can't be > 100% full of water)
IF Irrigation_Requirement < 0 THEN Irrigation_Requirement = 0

 

The user's irrigation program:

// Program 'Irrigate If Needed'
// The IF condition decides whether to water by comparing the Irrigation Requirement balance to a constant
// The user is responsible for setting this constant equal to Water Applied per Irrigation
// (and updating it if they change the value of Water Applied per Irrigation in their configuration)
If
       Time is 01:00:00AM
       AND Module 'Climate' Irrigation Requirement >= 0.5

Then
       Set 'Front Yard' On
       Wait  10 minutes
       Set 'Front Yard' Off
       Wait  3 seconds
       Set 'Back Yard' On
       Wait  5 minutes
       Set 'Back Yard' Off

       Trigger 'Reduce Irrigation Requirement'

Else
  - No Actions - (To add one, press 'Action')

// The final THEN action (after the irrigation zones all run) triggers ISY internal calculation of:
// Irrigation_Requirement = Irrigation_Requirement - Water_Applied_per_Irrigation

Please understand that in the absence of any proposed syntax for system commands within programs I have arbitrarily written the ISY code using a fictitious keyword "Trigger." I will happily edit my post as soon as Michel lets us know what the syntax should be.

 

Peter

[peterd]

A couple of quick notes on the preceding post.

 

I propose calculating Evapotranspiration_Yesterday once @ midnight because there doesn't appear to be anything which could be done with a partial-day average (except look at it in the UI), and it's just one less calculation to be done at each WB update (in my configuration that would amount to 2779 surplus calculations to every one used for an irrigation calculation).

 

To simplify everyone getting started using this, I've proposed not exposing the variables Water_Deficit_Yesterday and Evapotranspiration_Yesterday. Frankly, until there's floating point math in ISY programs (not a knock, just reality) these raw components of the irrigation decision calculation couldn't really be put to use.

 

I feel that if we give people a starting point in this next beta which aims to be as straightforward as possible, we'll begin to gain experience and get feedback. Nothing would prevent adding bells and whistles down the road (particularly once we have floating point variables).

 

Peter

[Michel Kohanim]

Hi Peter,

 

Thanks so very much! This will get us started rather immediately then we can make it more sophisticated as we go along.

 

Thanks again for your time and effort.

 

With kind regards,

Michel

[Mark Sanctuary]

Thanks Peter! This basic plan looks like it will be a good starting point.

 

Michel,

 

The plan up here did not mention the Trigger 'Reset Irrigation_Requirement' command, is this command still planned for the first pass?

 

I understand we are not to add anything to the Wiki that is not in the ISY ie. proposed code examples, so do you mind if we only work on the Wiki section about Evapotranspiration theory?

 

Thanks,

Mark

[peterd]

Michel -

 

I did not mean to omit the command to reset Irrigation Requirement to zero. (Just hadn't written up an example which used it.) I do believe we should have this from day one.

 

Thanks,

Peter