Is it possible to schedule ISY backups?

[dec3169]

I'm a Unix guy that uses cron extensively to do whatever I need done automatically. I also like to take backups of my systems regularly since I tend to tinker and often screw things up. I back up my ISY to a dropbox account fairly regularly, but I often forget.

 

Is there a way to automatically have the ISY do backups? Maybe something like cron or Windows scheduler, or a way to send the ISY a command to do the backup? I'd prefer a true scheduler on the ISY, but I could always have one of my Unix boxes send a string to the ISY from their cron job telling it to do the backup at a specified time if that ability existed. I didn't see that in REST, and don't know enough about the W or JSDKs to know whether it can be done in there.

 

Maybe I missed it if it is included. If not, does anyone have any suggestions or will something like this make it into the ISY as a feature? If so, it could be extended to do things like send the logs daily somewhere, or something else you can only do inside of the admin console. I searched the forums but didn't see this asked before.

 

Thanks,

Bob

[Exten]

2nd that notion...would be nice if it could auto backup and attach the backup

to an email and send it somewhere. Since the ISY can already do an email it

seems better than having to code up an ftp scenario or the like.

 

Just a simple check box...do a backup or not.

Then we make a program within the ISY to name the time and date

to do the backup and send the email. Ideally, the ISY would ship with a backup

program in place, similar to the factory query program.

The backup program would, of course, be disabled until you set an email addy

for it to send to.

 

Minimize the coding required - don't bother with a cron scenario

built into the ISY. The basic components are already there.

Email and a scheduler.

 

Maybe the default could be to backup on the SD card.

Then we change it to an email attachment.

[Michel Kohanim]

Hi Guys,

 

We have had this request many times and we have decided against it for the following reasons:

1. ISY does not need to be backed up unless you are explicitly making changes to its configuration. In those case, it's always best to backup at success points. So, in short, once you are done with any configuration, take a backup and you are done

2. The problem of automatic backup gets into many intricate details of delta vs. full, at what times, during what events, blocked by what other events, and many different permutations thereof

 

With kind regards,

Michel

[Xathros]

How difficult would it be to set it up so we could request a backup via a URL with wget or something similar?

 

-Xathros

[Exten]

I have to admit that manually doing a backup is a very easy thing to do when needed.

Of course, I can envision scenarios where an automated remote backup attached to an email

would be a nice thing (rental property when I am not there or on premises), but maybe the

payback isn't much in light of feature upgrades and bug fixes.

[johnnyt]

I have to admit that manually doing a backup is a very easy thing to do when needed.

but once you get to a few hundred programs, it's painfully slow (like loading programs). I'd like to be able to schedule a nightly backup - or better yet, be able to buy a faster processor and/or gigabit network speed.

[Michel Kohanim]

johnnyt,

 

Do you make daily changes to your programs and or ISY configuration? If not, then I am not sure why you would need a nightly backup.

 

With kind regards,

Michel

[apostolakisl]

johnnyt,

 

Do you make daily changes to your programs and or ISY configuration? If not, then I am not sure why you would need a nightly backup.

 

With kind regards,

Michel

 

How about a simple compromise. When you exit the admin console, a pop-up that asks you if you want to do a backup.

 

The OP complains that he "often forgets" to do backups, and this should fix that problem. I would also admit to be a bit lax about backups after making changes. But, as mentioned, backups serve no purpose unless something was changed. And nothing can change via any mechanism aside from a person logging in and changing something, so a system that reminds that person upon exit should cover you.

 

Conceptually, this is how Elk works. When you try to close the admin console, it asks you if you want to save it. Saving an Elk config is the same thing as an Elk backup since a "save" is a backup file of the Elk configuration, not the actual, uploaded to Elk, file.

[Michel Kohanim]

Hello apostolakisl,

 

Good idea with one exception: we have very many customers who use Admin Console only for monitoring. It might become annoying ... nonetheless, it's a good idea.

 

With kind regards,

Michel

[johnnyt]

johnnyt,

 

Do you make daily changes to your programs and or ISY configuration? If not, then I am not sure why you would need a nightly backup.

 

With kind regards,

Michel

Hi Michel,

 

Thanks for the question.

 

With a day job and 3 kids, I typically do my ISY programming in chunks of time over 3-4 days. Am currently migrating more stuff off HomeSeer (details here viewtopic.php?f=78&t=9840&p=74640#p74640) so working almost weekly in that fashion.

 

With 484 programs, 103 variables, 135 nodes, 71 scenes, and 13 network resources, I've had many chunks of 3-4 days or more of work over the past couple of years, each time running a backup that takes over 2.5 minutes (throughput of about 62KB/sec).

 

An automatic daily ISY backup, or an automatic backup on exit (with a way to turn it on/off?) would help if more processing speed (what I'd really like viewtopic.php?f=7&t=7768) is not an option. At 155KB per backup am not worried about wasting space on the days I'm not making changes. I might also point out that I do backups of 4 computers and 4-5 VMs on a daily basis and don't change stuff on them every day. Don't worry about a debate around incremental backups. That would only be of notable benefit if disk space was an issue, IMO.

 

edit typo: throughput is 62KB/min (1.0333 KB/sec)

[io_guy]

I don't know the capabilities of the ISY to comment, but what about just making the SD card contents available via FTP, RSYNC, SMB or some other file sharing method.

 

That way it's invisible to the admin console and power users, if desired, could schedule backups at any frequency they want using their own choice of software.

[apostolakisl]

Hello apostolakisl,

 

Good idea with one exception: we have very many customers who use Admin Console only for monitoring. It might become annoying ... nonetheless, it's a good idea.

 

With kind regards,

Michel

 

You could add one of those little check boxes in the corner that says "don't show this message in the future".

[Michel Kohanim]

Hi all,

 

Although quite simple at first glance, things get quickly out of hand:

1. Add a checkbox to see if the user wants to backup before exiting Admin Console

2. In case of automated, how does one notify of failures? What type of QOS should be used? Is it high priority, low priority, how many retries, etc.

3. As far as power users, I think that would be an option that we could consider

 

With kind regards,

Michel

[Exten]

Goodness....if there is an automagic checkbox for when you exit the admin console

then please please make that a preference. There is absolutely nothing more annoying

than having to tell the darned ISY....no durnit...if I wanted to back up I would have

done so...so shut up. I really don't want to be bothered to do a backup when I leave

the admin console. That's not ubiquity. That's not automatic if I must acknowledge

a dialogue box. Preferring to have a reminder is good. Pissin' me off continually

when I quit the admin console isn't.

 

Doing an automatic backup when leaving the admin console is 1 thing.

Prompting about the subject is quite another.

[Exten]

Hi all,

 

2. In case of automated, how does one notify of failures? What type of QOS should be used? Is it high priority, low priority, how many retries, etc.

3. As far as power users, I think that would be an option that we could consider

 

With kind regards,

Michel

 

A little green button that indicates successful last backup when you log in.

It would be red if unsuccessful.

That would be the minimum indication.

So if the email notification of successful backup failed, there would at least

be the green or red button to tell you it happened or did not.

 

The actual notification mechanism already exists. Email.

The scheduling component already exists too.

[johnnyt]

 

With 484 programs, 103 variables, 135 nodes, 71 scenes, and 13 network resources, I've had many chunks of 3-4 days or more of work over the past couple of years, each time running a backup that takes over 2.5 minutes (throughput of about 62KB/sec).

 

edit typo: throughput is 62KB/min (1.0333 KB/sec)

I just updated to 3.3.4 RC1 from 3.2.6 and a backup now takes 20 seconds longer (~13%) and adds about 4KB (compressed) for a throughput of about 56KB/min, or 0.94KB/sec. I don't know if it's all or mostly because of the file locking being done but would storing the data uncompressed make things notably faster? I'm not worried about disk space. I can always put the files in a compressed folder and let the core i7 in my system do the compression in milliseconds. My system backup solution also does compression.

 

If asked to chose only one, I'd pick a fast backup over an automated one since I do backups as I'm programming, which is when waiting close to 3 minutes seems like forever.

[Michel Kohanim]

Hi johnnyt,

 

We have made some improvements in 3.3.5. This said, I am not so sure it will satisfy your requirements as it seems your expecting ISY to be as responsive as a high powered computer.

 

With kind regards,

Michel

[johnnyt]

Hi johnnyt,

 

We have made some improvements in 3.3.5. This said, I am not so sure it will satisfy your requirements as it seems your expecting ISY to be as responsive as a high powered computer.

 

With kind regards,

Michel

Hi Michel,

When I copy a file using my high powered computer I get throughput of about 60MB/sec. A 155KB file would take a fraction of a second but that's not, in fact, my expectation. I will, however, admit that my expectation is closer to that than 3 minutes.

 

I like - make that love - that ISY is rock solid at doing the stuff I program it to do - and doesn't need regular reboots to keep going. There's also great support for it, as well as support for other insteon/electrical issues that frankly are Smarthome/other issues. I also find its impeccable logic refreshing. Once one has figured out it's nuances, it's amazingly powerful and predictable, although it does often require writing multiple programs to do one thing, making it a bit hard to follow and dangerous to mess with a year or more later. It was only after the ELK stuff was added (3.1.x) - even though I don't use it - and that I had reached about 300 programs that I did kind of hit my annoyance threshold for the time it takes to make a quick change, and to do backups when I'm doing more than quick changes.

 

Anyway, that's just me. Obviously if no one else is complaining, I'm just going to have to keep sucking it up. Everything comes with a trade off, doesn't it?

 

Thanks for the effort you do put into making ISY a solid product and addressing your community's interests.

[johnnyt]

I will also confess that part of my performance problems are because HomeSeer is subscribed to ISY and every time I make a programming change, I have to wait while ISY updates HomeSeer of what seems to be everything all over again.

 

I hope to one day get rid of HomeSeer but I will need ISY to support a wired thermostat (even if I have to buy a new one.) I don't really want a wireless thermostat because my furnace is on a UPS and I need to be able to send it commands when the power is out. Plus I don't trust wireless for this mission critical function.

[Michel Kohanim]

Hello johnnyt,

 

Throughput of downloading a large file from ISY is about 590 KB/S under heavy load ... closer to 700 KB/S with no load; about 200 KB/S using AES128/https 1024. The problem is that programs are not compressed into a large file. Admin Console is actually downloading individual programs in which case you also need to take into account TCP connect times as well.

 

In any case, hopefully our performance improvements are going to help a little.

 

With kind regards,

Michel