Hard drive crash, the third in ten years, yet we never lose data. Several years ago I whipped up a simple backup system that doesn’t use The Cloud1 or expensive backup hardware, only three flash drives that cost maybe $10 each.
Backup system components
Our home network has two Apple MacBooks so this backup system is Apple-centric.
Encrypted sparse bundle
The heart of the system is a special directory called a sparse bundle. The directory is broken into bands so it isn’t a big monolithic blob of data. This allows you to back up only the bands that have changed rather than the whole blob.
I created on my laptop an encrypted sparse bundle named oddments.sparsebundle. On the other laptop I created a directory named backups.
The backup script uses rsync to copy the sparse bundle bands that have changed since the last backup. Rsync speeds up the backup process nicely because only the data that changed gets copied. Otherwise I’d be copying old data that never changes, like photos and pdf files, every time I back up.
Three flash drives
The 3-2-1 backup rule: keep three backups of your data on two different storage types, with at least one backup offsite.2
I use three flash drives: the drive with the latest backup goes in my pocket, the second latest goes in my car, and the oldest goes in my study. Just before running the backup script I move the pocket drive to the car, the car drive to the study, and the study drive is plugged into the laptop. Then the script is run.
The script, explained
I generated an RSA key pair on my laptop and installed the public key on Penny’s laptop. Now I (and the script) can ssh into her laptop without entering a password.
- Mount the encrypted drive image to /Volumes/oddments.
- Rsync Penny’s Documents folder to /Volumes/oddments/penny.
- Rsync Penny’s Pictures folder to /Volumes/oddments/penny.
- Rsync my important folders to /Volumes/oddments/jake.
- Unmount the encrypted drive image.
- Compact the encrypted drive image, otherwise deleting old data won’t shrink the image size.
- Rsync the encrypted drive image to Penny’s computer.
- Rsync the encrypted drive image to the flash drive, if installed.
So we end up with our important files copied to her computer, my computer, and a flash drive. Everything is encrypted so if the flash drive is lost, no one can read the data.
Note: Data like movies, music, and podcasts are ignored because 1) they take up a lot of space 2) they’re backed up in other ways such as CDs, DVDs, or in Apple iTunes. This keeps the final encrypted drive image fairly small.
#!/bin/bash HOST=penny cd /Users/jakehoward echo "Mounting encrypted drive image" hdiutil mount oddments.sparsebundle echo "Backing up Penny Documents" rsync -amz --delete --delete-excluded --exclude .DS_Store -e ssh $HOST:Documents /Volumes/oddments/penny echo "Backing up Penny Pictures" rsync -amz --delete --delete-excluded --exclude .DS_Store --exclude 'iPod Photo Cache' --exclude "Photo Booth Library" --exclude *.photoslibrary -e ssh $HOST:Pictures /Volumes/oddments/penny echo "Backing up my laptop" rsync -am --delete --delete-excluded --cvs-exclude --exclude *.pyc --exclude .DS_Store --exclude 'Photo Booth Library' --exclude "iPod Photo Cache" --exclude *.photoslibrary --exclude build/ --exclude dist/ .profile .ssh .hg* .vim .vimrc Documents Pictures scripts projects /Volumes/oddments/jake echo "Unmounting encrypted disk image" hdiutil detach /Volumes/oddments echo "Compacting encrypted disk image" hdiutil compact -batteryallowed oddments.sparsebundle echo "Copying encrypted disk image to Penny's computer" rsync -avzr -e ssh oddments.sparsebundle $HOST:backups/ if [ -d /Volumes/flashdrive ] then echo "Copying encrypted disk image to thumb drive" rsync -amvr oddments.sparsebundle /Volumes/flashdrive/ echo "Compacting encrypted disk image on thumb drive" hdiutil compact -batteryallowed /Volumes/flashdrive/oddments.sparsebundle else echo "Thumb drive not detected" fi
Why not use cloud storage?
Cloud backup and recovery is so slow that one cloud backup company snail mails your recovery data on a hard drive. It’s cheaper and faster than downloading it at 50Mbps for days.