should I avoid writing to SD Card when I can?

[johnnyt]

Have read a couple of posts that suggest or seem to suggest avoiding writing to non volatile memory. I have several programs I use as variables, many created before variables were available and many simply because I got used to that paradigm. In the then clause I "enable run at startup" and in the else clause I "disable run at startup" to preserve the state through a reboot. If I was using variables I would use the "init to" command to do the same thing, which also writes to nvram I believe.

 

Am aware of some of the drawbacks with solid state disk and ran into what I now think was a performance problem due to perhaps a lack of garbage collection on it (didn't really get an answer when I posted on that but it might be because it was near the bottom of a long post. see http://forum.universal-devices.com/viewtopic.php?f=7&t=7768&p=60001&hilit=garbage+collection#p60001)

 

What are some recommendations when it comes to this?

[bmercier]

Hi Johnny,

 

My take on this subject is this;

 

Whenever persistence is not required, it's preferable to avoid writing to nvram. Not because of speed, but rather longevity of the SDCard. Maybe it's negligible if you write to it only once in a while. But if you have programs that writes "frequently", I would try to avoid it if possible. As you guessed, enabling/disabling programs, or using InitTo on variables is not better. But if you don't need persistence, use variables without InitTo.

 

As far as speed is concerned, at execution time, writing to nvram takes longer that writing to the ram, but it's really negligible unless a program is looping continuously.

 

Your post seems to suggest that writing often to nvram causes performance issues in the long run. I'm not sure how this file system works, and if this could cause things like disk fragmentation. Maybe changing the enable/disable flag writes a new file, maybe it stores it's data on the nvram in another spot than the file system, I really have no clue on this one.

 

Benoit.

[io_guy]

I would doubt you'd see any "garbage collection" impact in this application. SD cards are extremely slow to begin with and there is very little fragmentation impact since the files/data you're reading is tiny anyway.

This really comes into play with things like SSDs where you're expecting 100-500MB/s (depending on model) and large files.

 

Wear-leveling, or wear in general would be the bigger concern, but it will likely be low as well. Virtually every memory controller these days has proper wear-leveling and with the amount of free space on our ISY cards, there a lot of room to play musical chairs.

 

This used to be a problem years back. My automation house used to build embedded industrial PCs which used CF for storage. These controllers had no wear-leveling. With an ~1,000,000 (very rough number) write before death limit, a card would die in less than 2 weeks if you wrote to the same spot every second.

 

That being said, avoid writing wherever possible. In a "real-time" system (or a small automation controller with limited resources) it is bad practice. If you're concerned about saving data, stick the ISY on a cheap battery backup:

http://1wt.eu/articles/alix-ups/

 

Just my ramblings...

[johnnyt]

Thanks for the replies. I will re-evaluate my need for persistence in my programs.

 

As far as speed is concerned, at execution time, writing to nvram takes longer that writing to the ram, but it's really negligible unless a program is looping continuously.

Interesting. I think I may have evidence that there is a visible performance problem related to writing to nvram, at least in more recent firmware releases. I've found that after I moved to 3.1.16 from 3.1.11 I needed to add a "wait" after a "send notification" command and before updating a state variable and saving it. Otherwise I would get the standard notification instead of the custom notification I was supposed to get. I was told it's because the system is busy but it happens every time without fail so the busy condition seems to be writing and saving a state variable. See http://forum.universal-devices.com/viewtopic.php?f=25&t=7693&p=58831#p58831

[Michel Kohanim]

johnnyt,

 

I do apologize for not getting back to you sooner. Garbage collection means nothing to us because we manage the whole memory. We actually know where each object is in the memory.

 

As far as SD card read/write, io_guy is 100% correct and we do have wear_leveling. We have systems out there that have been running for more than 3 years without reboot.

 

As far as notifications, whenever you change the state variable, all programs depending on it are re-evaluated. Can you copy and paste the program for which you have problems with notifications?

 

With kind regards,

Michel

[johnnyt]

Garbage collection means nothing to us because we manage the whole memory. We actually know where each object is in the memory.

If not a garbage collection issue, any idea what would have caused my SD card to begin impacting performance the way it did without actually failing or showing any bad blocks?

 

 

As far as notifications, whenever you change the state variable, all programs depending on it are re-evaluated. Can you copy and paste the program for which you have problems with notifications?

Sure, here it is:

 

If

$sLightOut is 1

And (

Time is Sunset - 30 minutes

Or (

From Sunset - 1 hour and 30 minutes

To Sunset - 29 minutes (same day)

And Control 'MISC (Non Lighting) / Dusk and Low Bat Sensors / Front Door Dusk-Dawn' is switched On

)

)

 

Then

Send Notification to 'JT' content 'Dark Out'

$sLightOut Init To 0

$sLightOut = 0

 

Else

- No Actions - (To add one, press 'Action')

 

I have about 4 other programs that would re-evaluate

[Michel Kohanim]

Hi johnnyt,

 

Thank you. I think it would be best for you to send all those programs impacted by this variable to support@universal-devices.com. There's basically nothing wrong with this program and I am very much concerned that there are things happening in your system causing it to be slow.

 

In the meantime, when you have slow response times, can you please pay attention to the activity on the Mem light on ISY? That would give us an indication of how much activity there is on the file system. I am still thinking that there might be some tight loops somewhere causing all the performance issues.

 

If all else fails, I would like to login to your system remotely and see what's going on.

 

And, a final thought, in most systems the bottle neck is usually the file system (including on all computers except perhaps SSDs). As I mentioned before, ISY puts a lock around each file and thus if you have multiple tasks requiring the lock, obviously there's going to be a performance hit.

 

With kind regards,

Michel

[johnnyt]

Looks like the HomeSeer ISY plug-in is the cause of my performance problem with notifications, not the SD card. see http://forum.universal-devices.com/viewtopic.php?f=27&t=7939&p=60301#p60301

 

I'm still not clear why replacing my SD Card did improve my performance in some areas (with HS plug-in running) and the notification problem didn't occur with firmware 3.1.11 and before.