Proposed Evapotranspiration Algorithm in 3.1.4 Beta

[Mark Sanctuary]

Peter,

If Michel confirms, just add a separate section on the bottom of your existing proposal, maybe something like this.

 

[...]

 

Module 'Climate' Commands:

// A user program command that executes the Irrigation_Requirement formula
Trigger 'Reduce Irrigation Requirement'

// A user program command that resets the Irrigation_Requirement to zero
Trigger 'Reset Irrigation Requirement'

Please understand that in the absence of any proposed syntax for system commands within programs I have arbitrarily written the ISY code using a fictitious keyword "Trigger." I will happily edit my post as soon as Michel lets us know what the syntax should be.

[Michel Kohanim]

Mark/Peter,

 

Yes, we will have those commands. I feel a lot better about this solution.

 

Mark, as long as the following norms of Wiki are adhered to, then you are welcome to post:

1. No blah blah and no profanity

2. Fact based:

--a. What ETo means is a fact

--b. What can currently be done with ISY capabilities are facts

--c. What ISY may or may not do in the future are not facts

 

Thanks again and with kind regards,

Michel

[Mark Sanctuary]

Great! I will work on it. Sorry if you have mentioned this before and I forgot. I did not mean to added posts that where unacceptable. Sometimes my memory fails me especially when I hang at the forums then get busy at work and return here six months later to hang again. Let me add these rules to the community section of the Wiki so I can reference them in the future.

[Michel Kohanim]

Hi Peter,

 

I think we are missing the water applied in the following equation:

Water_Deficit_Yesterday = Evapotranspiration_Yesterday - (Rain_Yesterday * (Soil_Absorption_Factor / 100))

 

Should be:

Water_Deficit_Yesterday = Evapotranspiration_Yesterday - ((Rain_Yesterday+(Water_Applied_Per_Irrigation*Num_Irrigation_Cycles_Yesterday)) * (Soil_Absorption_Factor / 100))

 

The reason is that if we simply decrement Irrigation_Requirement by Water_Applied_Per_Irrigation, we are simply ignoring the Soil_Absorption_Factor. In short, we should have total water applied (rain + irrigation cycles) * Soil_Absorption_Factor/100.

 

Don't you think so?

 

With kind regards,

Michel

[peterd]

Michel -

 

I understand your point, but I'm not sure about implementing it that way. 3 reasons:

 

1. I believe Water Deficit is a "pure" number not reflecting any human intervention. (That is, it should provide a number representing only the effects of weather and the plants themselves on the water content of the soil.) In other words, the Water Deficit is always the number before any irrigation happens, but the Irrigation Requirement balance reflects the effect of irrigation as soon as the user's program says "ok I've watered." My reasoning has to do with having this data available to programs in the future (once they could perform math on it).

 

2. I included soil absorption in the Water Deficit calculation as a way to approximate its effect on rainfall, because there would be no other way for users to factor it in. However, in the absence of either a fully-featured irrigation module or enough programming features (including floating point variables & math) to move those calculations into user code, a single number here may be a big compromise.

 

In Mark's case, for example he has 2 different soil types. I am in the process of landscaping a newly constructed hillside which has drainage incorporated underneath to prevent erosion - it may have a very different absorption rate than the rest of my property. (I can measure this empirically once it's complete.) In cases like these, users would adjust zone timings accordingly, effectively incorporating multiple absorption rates into the irrigation schedule (over which - unlike rainfall - they have control).

 

3. I'm not sure what are all the implications of moving the Water Applied decrement from the programmatic trigger point, and of the completion of irrigation no longer having a direct and immediate effect on the Irrigation Requirement balance? What happens when the user resets the balance to zero? Does Water Deficit change (i.e. does Irrigation Cycles go from 1 to zero even though an Irrigation Cycle in fact happened)?

 

 

All that having been said, I also understand the desire to simplify things even further for the user. However, at the moment I believe that (just as today) the user will have to figure out how much water to deliver to each zone in order to achieve the appropriate replenishment for different plant types (root depths & water uptake rates) and soil types.

 

For someone with any degree of complexity in their landscaping (i.e. more than just a lawn), I'm not sure how to make this initial programming easier. On the other hand, I'm concerned that trying to eliminate something from the users' one-time zone calculations may help those with vary basic needs (for whom the zone timing may be much simpler to begin with) while hampering those with sophisticated setups (such as my 20 zones) - for whom this whole ET capability may have the most benefit.

 

I know that's a lot to consider at a time when you'd probably prefer a yes/no answer. Let me put it this way:

If we don't implement this now, users will factor it into their zone timings directly. Its presence might prevent users from doing what they need to on a zone-by-zone basis, but its absence should not. We could always add it later.

 

Peter

[Mark Sanctuary]

Peter,

 

If my thinking cap is on correct... Should we just use the soil factor on both sides then the user would not have to do any manual soil factoring but rather just deal with ins and outs of the water by adjusting the length of runtime per station?

Water_Deficit_Yesterday = Evapotranspiration_Yesterday - (Rain_Yesterday * (Soil_Absorption_Factor / 100))
// Applied Rain water is factored by soil type and addresses if the soil is more/less porous and looses its water faster/slower

Irrigation_Requirement = Irrigation_Requirement - (Water_Applied_per_Irrigation * (Soil_Absorption_Factor / 100))
// Applied Irrigation water is factored by soil type and addresses if the soil is more/less porous and looses its water faster/slower

The only bummer I can think of for now would be the different soil types per station, but we are already kind of half way in this issue since we are only able to do one kind of factoring on Rain_Yesterday with our current design anyways.

 

Thanks,

[IndyMike]

Hello Gents,

 

I'm finally back from my little work visit to the LA area. I'll freely admit that I have not been able to keep up with the numerous posts since my departure. As a result, much of what I am posting may no longer be pertinent.

 

Prior to my departure, I set the ISY to log data from the nearby Airport NAOA Class I weather site. I must admit, I'm a bit underwhelmed by the data compiled. Nonetheless, it does point out some possible problems in implementing the ETo calculations.

 

From the table below:

1) The first and last ROWs of the table are listed as being the first data past midnight. The weather station should have reset it's high/low and rain data. The data (highlighted yellow) is obviously from the previous day.

2) The "Rain Today" column has multiple instances where the rain total reduced as the day went on.

3) I have "gaps" in the reporting where time intervals were missed (9:11:10 missing). Over a period of 12 days, I do not have a single day that has all 24 measurements. Note: this is being reported to Email. It's possible that this is an email problem rather than a weatherbug problem.

4) Wind speed and Average Wind speed - identical throughout the day. Not sure which is being reported, but one of these is incorrect.

5) Light %: Not reported. This really hurt. I was hoping to use this as an estimator for Ra.

6) Low Temp (red highlight): My mistake - this info is correct.

 

 

Using the information in the UCDavis document (http://biomet.ucdavis.edu/Evapotranspiration/PMhrXLS/PMhrDoc.pdf) I constructed a spreadsheet to tabulate the hourly ETo based on my recorded data. All of the hourly calculations that I have found want a real time estimate of Rs (net shortwave radiation). I've pulled archived data for my area (2005) for both cloudy and sunny days. Since we received 1.7 inches of rain in the table presented above, I believe that Jun 9th could be classified as a cloudy day.

 

The following is a comparison of the ISY calculation vs my spreadsheet (cloudy and sunny days). If you can't read the legend, The ISY report is in Red, the clear day calculation in black, and the cloudy day in Violet. Note that I have disregarded the 12:11 am data (previous day) in my calculations.

 

 

I found it interesting that the ISY predicted the highest ETo at 1 am. I'm not sure whether this is a result of the erroneous 12:11 data indicating that the high for the day was 95 degrees. Michel had previously indicated that there was an error in the ETo calculation where the high temperature(??) was being used as the current temp. If that were the case, I would have expected a nearly flat line ETo calc.

 

Since my return, I've re-vectored the ISY to the Weatherbug station at the local high school. Data is reported in 5 minute intervals and the Windspeed and Light% appear to be operating correctly. I'll try this again in a few days time to see if the calculations have improved.

[peterd]

Mike -

 

Thanks! Michel, Mark & I have been working on the overall application of ET averages to automate irrigation. But, yours is exactly the kind of analysis to validate & debug the base calculation which we very much need.

 

I'll be eager to see how your next round (with the other WB station) goes. The available stations seem to vary greatly in terms of what data is available and at what interval. I found that my local NOAA station only reported hourly & had no light %. One of my local WB stations reported all rain values as "N/A" (the ISY wasn't too happy with this) while another was down for an extended period. That last one is finally back up and all values seem to be available, and as instantaneous as I care to have (I can set a polling interval under 1 minute & see values changing). Fingers crossed that this station continues to be operational.

 

While I have not done any rigorous observation or analysis, the current instantaneous ET calculations in the ISY appear to be higher than historical averages for my area at this time of year (by about double).

 

Cheers,

Peter

[Michel Kohanim]

Peter,

 

Thank you. We have most of the application done.

 

I agree with Mark: we should apply the soil factor to each irrigation run. If we have different soil types, then we have to make the application to use different zones/soil types.

 

IndyMike, as I mentioned before, the current ET0 calculations in ISY have a bug (it uses the High Temp all the time instead of average) and thus the high ET0 reported rates.

 

With kind regards,

Michel

[peterd]

Peter,

 

Thank you.

Happy to help.

 

[...] we should apply the soil factor to each irrigation run [...]

 

Michel -

 

I assume you mean as Mark illustrated:

Irrigation_Requirement = Irrigation_Requirement - (Water_Applied_per_Irrigation * (Soil_Absorption_Factor / 100))

 

That's fine, but the user's irrigation program has to take this into account:

// Program 'Irrigate If Needed'

// The IF condition decides whether to water by comparing the Irrigation Requirement balance to a constant

// The user is responsible for setting this constant equal to Water_Applied_per_Irrigation * (Soil_Absorption_Factor / 100)

// (and updating it if they change the value of Water Applied per Irrigation or Soil Absorption Factor in their configuration)

 

So if my previous example, for a Water Applied per Irrigation depth of 1/2 inch had been:

IF
       Time is 01:00:00AM
       AND Module 'Climate' Irrigation Requirement >= 0.5
THEN
       ...

And I've got Loam sand (83% Absorption Factor), then my code will read:

IF
       Time is 01:00:00AM
       AND Module 'Climate' Irrigation Requirement >= 0.415
THEN
       ...

 

To me, that makes the user's code even more brittle (because now we're duplicating in code not just a value directly entered but one which is derived by a non-obvious internal calculation), and the whole concept much more difficult to understand/explain.

 

Peter

[Mark Sanctuary]

Peter,

 

IF
       Time is 01:00:00AM
       AND Module 'Climate' Irrigation Requirement >= 0.5
THEN
       ...

And I've got Loam sand (83% Absorption Factor), then my code will read:

IF
       Time is 01:00:00AM
       AND Module 'Climate' Irrigation Requirement >= 0.415
THEN
       ...

What I was thinking we could just adjust runtime to round off the irrigation_requirement check numbers... we can instead of applying 12 minutes for the 0.415 we can do say 14 minutes for the 0.5 application. This way the soil type that takes more to saturate with water gets more water.

 

(this is only illustration, I did not go and actually calculate the runtime so just keep this in mind)

 

Does that work?

[IndyMike]

Michel,

 

I do remember you stating that the ET equation was using the high temperature instead of the current reading. From what I can see, that's one piece of the problem.

 

My post was really concerned with the quality of the WB data. The data that I recorded from the local NAOA site was simply garbage. Unfortunately, when it comes to ET calculations, Garbage in = garbage out. Many of the errors posted above could be circumvented in software (weather data not resetting after midnight), but you need to know of the error in order to account for it. I fear that rolling out a ET calculation to users may create gigantic headaches for the ISY team if the WB data input has errors similar to what I posted above.

 

Beyond that, I would humbly submit that the ET calculation has additional errors beyond the use of the high temperature instead of the current temp. I simply can't accept the fact that the maximum ETo occurred at 1 AM with no sun and 63% humidity. It doesn't make sense.

 

I played a bit more with my calculators in an attempt to match the ISY predictions over this period. Given the error you noted, I used a min/max/current temp of 95F for the entire 24 hours. This still did not approach the values predicted by the ISY.

 

I was able to come close to the prediction by using a 24 hour Rs value of 400 W/M^2 (1.44 Mj/m^2/Hour). This would definitely qualify as the sunniest day in South Bend history. The fact that it occurred on a day when we received 1.72 inches of rain is even more troubling.

 

The plot below shows the ISY calculation VS 3 different simulations. The plots are presented as accumulated ETo over a 24 hour period starting at midnight. As I stated previously, the closest simulation was calculated with 24 hour sunlight and a constant 95F temperature. The other two simulations were performed with "typical sunny" and "typical cloudy" days. These are correlated with the "daily" calculator that I referenced previously (single data point at the 24 hour mark). The cloudy day calculation (blue line) is likely the closest to reality.

 

 

I am really not trying to throw stones here. I genuinely want to help progress this to a workable tool. If there is something I can do to help in evaluating your algorithm, please let me know. If I'm being a PITA, let me know that as well.

 

IM

[peterd]

[...]What I was thinking we could just adjust runtime to round off the irrigation_requirement check numbers... we can instead of applying 12 minutes for the 0.415 we can do say 14 minutes for the 0.5 application. [...]

 

Does that work?

Maybe, but I think you're actually making my point for me. Until we have either a full-featured module or all the tools (e.g. floating point vars & calcs) to do this all in user code, why not keep things as simple as possible?

 

I only included the soil factor because otherwise there'd be no way to take it into account for rainfall. OTOH, we can adjust the runtime - not to round off the Irrigation_Requirement check numbers, but to directly account for the soil absorption (and do that zone-by-zone). If the user figures out the effect of soil absorption on runtime (e.g. "running this zone for 14 minutes will result in 0.5 inch being absorbed") as part of their one-time setup calculations outside of the ISY, then the comparison on a day-to-day basis remains easy to understand ("When Irrigation Requirement >= Water Applied per Irrigation, then I apply Water Applied per Irrigation").

 

Peter

[Mark Sanctuary]

Peter,

 

I think all you have to do is calculate the runtime offset with an online javascript app (I plan on creating one that is just for ISY use tailored to the user's needs).

 

Does water get absorbed any differently between rain and irrigation? It enters the soil and spreads throughout the particles. The only difference is rain has 100% spread and the dipper/sprinkler has a constrained square footage spread. So why would that entry be any different from rain vs dipper/sprinkler? Shouldn't soil type be set the same for all incoming water types?

 

Thanks,

[peterd]

Mark -

 

The issue is not that irrigation water is absorbed differently. It's what we're measuring. I simplified out some of the explanation to help move things along, but there is a concept called "Effective Rain." That's what we're calculating when we factor soil absorption into rainfall.

 

On the other hand, we haven't been dealing with "Effective" irrigation. Water Applied per Irrigation is not Water Absorbed per Irrigation. Hindsight being 20/20, if this variable had been named something like "Decrement from Irrigation Requirement Balance per Irrigation Run," perhaps it would have made the intent clearer.

 

As for soil type being entered differently - because we're trying for a basic initial implementation we're specifying a single value to approximate Effective Rain. You yourself indicated that you have multiple soil types. When you must compromise (ISY's internal calculations), that's one thing. But why would you want this being calculated for you when you can do this yourself (as part of the initial calculation of run-times which you must do anyway) exactly as needed for your situation?

 

Then there's the lack of clarity introduced by incorporating it into the behind the scenes calculations. If it were possible, the most logical, intuitive and understandable way to write the irrigation program would be:

IF
   Irrigation Requirement >= Water Applied per Irrigation

but that isn't possible. Therefore, we take one step away from the intuitive by having to specify the value of Water Applied per Irrigation in the program.

 

Having this constant be different than the value of Water Applied per Irrigation just makes it even harder to understand...

 

Peter

[Mark Sanctuary]

Then we should call it Effective_Rain_Factor which is what it is and would take away much of the confusion.

[peterd]

Then we should call it Effective_Rain_Factor which is what it is and would take away much of the confusion.

Works for me.

 

Michel -

 

Have you been following this part of the discussion? The proposal on the table is to rename "Soil Absorption Factor" as "Effective Rain Factor" and use it only in the rainfall side of the calculation. That way we make it clearer what we are calculating, and leave as many options as possible for enhancement going forward...

 

Peter

[Michel Kohanim]

Hi Peter, no, I have not.

 

I am not sure how Rain relates to Applied water. We updated the configuration UI to show the Actual water applied per cycle to make it easier for the user to know how much water is actually being used based on the soil type. We should have beta ready soon.

 

With kind regards,

Michel

[Mark Sanctuary]

Great news Michel!

 

I am excited to see what you guys have built from all this intense deliberations we have had on this subject. I think a BIG thanks is in order for all the hard work you and team have done on this module!

[Michel Kohanim]

Hi Mark/Peter, and a big thanks to you guys.

 

Sneak Preview:

[peterd]

Michel -

 

Thanks for the preview. It's exciting!

 

One clarification (maybe for beta 3.1.5 ):

- If you're going to apply the Soil Absorption Factor to irrigation water, then the label "Actual Water Applied per Irrigation Cycle" should be changed to "Allowable Depletion."

 

- The user should enter "Allowable Depletion", and be shown "Water Applied per Irrigation Cycle" (So in the UI, the calculation

(100/Soil_Absorption_Factor)*Allowable_Depletion

would be performed).

 

Here's why:

 

"Allowable Depletion" is the proper term for the amount of water loss from the soil before irrigating. (We've been referring to "Water Applied per Irrigation Cycle" as though it is synonymous with "Allowable Depletion" (which it was until the Soil Absorption Factor started being applied).)

 

Using the industry standard soil moisture balance method of irrigation management, the depth of plant roots is used to determine the ideal water depth. Some percentage of depletion (usually 50%) is allowed before watering to make up the difference. So, the Irrigation Requirement is tracked, and once the irrigation requirement reaches the allowed moisture depletion level it is time to irrigate.

 

So, as a concrete example, if our ideal water depth is 30mm and we want to water when half of that has been depleted by evapotranspiration, our Allowable Depletion is 15mm. That is also the amount of water we want absorbed by the soil. So if our Soil Absorption Factor were 83%, we'd need to apply 18.0725mm to result in 15mm absorbed. But, our ISY program will read:

IF
   Module 'Climate' Irrigation Requirement >= 15mm

 

As you've shown it, the user will either have to calculate the inverse of 83% and apply that to 15mm to determine that (s)he must enter 18.0725mm or sit there trying values until they get 15mm for the Actual Water Applied per Irrigation Cycle (Allowable Depletion).

 

And, when the "Irrigation Cycle Complete" action occurs, the ISY must decrement Irrigation Balance by Allowable Depletion (in this case 15mm) since that's what we've replaced with our irrigation cycle.

 

Cheers,

Peter

[Mark Sanctuary]

Peter,

"Actual Water Applied per Irrigation Cycle" is a read only GUI widget in this screen-capture so he is just displaying data.

 

 

Michel,

Are you just showing us what the number without "Absorption Factor/Soil Type"?

Actual_Water_Applied_per_Irrigation_Cycle =  Water_Applied_per_Irrigation_Cycle * (Soil_Absorption_Factor / 100)

Thanks,

[peterd]

Peter,

"Actual Water Applied per Irrigation Cycle" is a read only GUI widget in this screen-capture so he is just displaying data.[...]

Mark -

 

I understand. My point is that if we're going to enter one value & see another derived value, the value we need to be able to enter directly is Allowed Deficit ("Actual Water Applied per Irrigation Cycle").

 

Working backwards from how much water the soil can hold and how much we need to replace, we wind up with the number representing Allowed Deficit (i.e. 50% loss of 30mm total = 15mm). We were thinking of that number as "Water Applied per Irrigation Cycle" which is no longer correct if ISY is doing the Soil Absorption math on our irrigation. The user now needs to calculate out that ISY calculation in order to determine when they've applied enough water. In that example it's not helpful that ISY shows 12.45mm when they enter 15mm, because they need to figure out what to enter so that that "Actual" field displays the Water Deficit of 15mm (18.0725mm Water Applied per Irrigation Cycle = 15mm Water Deficit).

 

I tried to explain that there were implications to changing the algorithms I proposed, but wasn't successful. This is one of those implications.

 

I also really wish everyone involved had invested an hour to read over and understand at least the first couple of chapters of that PDF I linked to early in the thread because I feel we've spent far more time than that trying to explain/understand concepts, terminology and techniques which are well established among irrigation professionals who use ET.

 

This is not to say that I'm not very grateful to Michel & co. for soliciting our input (and to you for your participation) and for implementing ET at all, just that I don't think we're quite there yet on this one point.

 

Cheers,

Peter

[Mark Sanctuary]

So we need to do this...

Then we should call it Effective_Rain_Factor which is what it is and would take away much of the confusion.

Works for me.

 

Michel -

 

Have you been following this part of the discussion? The proposal on the table is to rename "Soil Absorption Factor" as "Effective Rain Factor" and use it only in the rainfall side of the calculation. That way we make it clearer what we are calculating, and leave as many options as possible for enhancement going forward...

 

Peter

And making this...

Irrigation_Requirement = Irrigation_Requirement - (Water_Applied_per_Irrigation * (Soil_Absorption_Factor / 100))

Be this...

Irrigation_Requirement = Irrigation_Requirement - (Water_Applied_per_Irrigation)

[Mark Sanctuary]

I also really wish everyone involved had invested an hour to read over and understand at least the first couple of chapters of that PDF I linked to early in the thread because I feel we've spent far more time than that trying to explain/understand concepts, terminology and techniques which are well established among irrigation professionals who use ET.

Peter,

 

I have read it many times, thinking about each thing, the hard part here is we have to all learn and discuss this over text. It's not like in a college class were we all went home studied it, then the next day got to have the teacher go over it again and then let the class discuss it. Also I would guess this topic would be broken into many parts over several days so it could be digested over several weeks. I think the time we have spent here is reasonable for what we are all trying to learn.

 

Thanks,