Synology Disk Station Off-site Backup

How to Arrive a Good (or optimal) Backup Solution?

The Synology Disk Station: DS111, with a single 1TB disk.
It is used as the network file system. All meaning files are stored on it.

Current Backup and Why Changing?

We currently use rsnapshot to back up the files on the DS111 to a separate folder, and store it to the PC upstairs, in case the hard drive fails. There are a few limitations.

1. The PC upstairs are often turned off, or goes to sleep. So we may end up of having not so recent backup if the DS111 disk fails.
2. If there is a catastrophe (hopefully never), there is no offsite backup

The Options

The major need is an affordable offsite backup. We considered these options.

1. Buy an external USB drive, plug it in occasionally for a backup, and take the drive offsite. It is very affordable, at as low at $56 for a 1 TB external USB drive. The main downside is that it can only be done infrequently, the hassle with "doing it", and most likely the reality of "not doing it". Also the USB 2.0 port in reality can only write at 25-30 MB/s, a full backup for 100GB can take about an hour to complete. For a 1 TB drive, it can take 8-9 hours.
2. Another choice is use a cloud based backup solution. A few criteria are necessary
1. Needs to be cost effective, like 1/3-1/4 of the cost of an external hard drive for a year

Considerations:

• Space: The folders that need to be backed up (homes, music, photo, web, public) add up to about 50GB (30k files). This can grow since:
• VHS tapes that need to be converted - about 5. There are many technical considerations. This link has information. size TBD
• miniDV tapes - about 12: One hour (standard size) video amounts to 13GB of raw data, once H.264 encoded, it comes down to about 4GB. This will lead to about 48GB total.
• Natural growth with photo and video. Other miscellaneous files can be ignored.
• So it is easily assumable the eventual size with the video will be about $100GB.
• Accessibility: The backup files should be accessible both from the Synology station, or other ways like the web site.
• Restore flexibility: Can restore all files, a set of files, or individual files.

Synology Cloud Backup Comparisons:

Synology supports cloud based backups. From the US, you pick either Microsoft Azure, or Amazon S3. Other one are more expensive. Based on the AWS S3 pricing

For 50GB, 30k files --

• S3 Standard: size, request: 50*0.03 + 0.005*30 = $1.65/month
• S3 Reduced Redundancy: 50*0.024 + 0.005*30 = $1.35/month
• S3 Glacier: 50*0.01 + 0.05*30 = $2/month

Noted that if backup is incremental, subsequent backups should be lower than the initial. However, there will be more than (1) backup events/month. So the total requests need to be modeled.

For 100GB, 40k files (adding video files, bigger, but not as many)

• S3 Standard: size, request: 100*0.03 + 0.005*40 = $3.2/month
• S3 Reduced Redundancy: 100*0.024 + 0.005*40 = $2.6/month
• S3 Glacier: 100*0.01 + 0.05*40 = $3/month

Even assume a bigger capacity, like 200GB, 50k files

• S3 Standard: size, request: 200*0.03 + 0.005*50 = $6.25/month
• S3 Reduced Redundancy: 200*0.024 + 0.005*50 = $5.05/month
• S3 Glacier: 200*0.01 + 0.05*50 = $4.5/month

So the S3 Reduced Redundancy tier is the lowest cost until the size gets closer to 200GB.

There is also the possibility of a hybrid approach, using S3 Reduced Redundancy for smaller files (possibly more often access), and S3 Glacier for media (video/audio) files.

A Hybrid Approach:

• Consider that the mutable files (homes ) is only 2 GB, with 2k files. These files are best backed up with S3 (Standard or Reduced Redundancy), for easy access as necessary. Even if these files grow to 100 GB, it only cost $3.2 or $2.6 per month.
• All the rest are immutable files (web, music, photo, and public), which means they will be uploaded once, deleted (rarely), and hopefully never or rarely retrieved. So the major cost driver will be the storage, instead of access. Taking 100 GB as example with S3 Glacier, the first month may cost $3 due to the initial load, but later will only cost about $1/month due to limited or no updates.

This hybrid approach will likely be the cheapest way to achieve the archive.

************** 8/14/2015 Update **************

It turned out that the "necessary number of" media files is much smaller than what it seems. Synology creates index thumbnails for the media files (5 per file). So the actual number of media files is only 1/6th of the apparent number. This is great in reducing the cost for Glacier backup.