A backup you don't have to think about is a backup that gets done
Backups are an easy task to put off but WOW do you feel the pain when a file is mistakenly deleted or a storage device fails!
I use a combination of rsync + ssh + rdiff-backup + cron to perform an automated daily backup of my desktop home directory to my home server.
As per the rdiff-backup
manpage ...
The target directory ends up a copy (mirror) of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves symlinks, special files, hardlinks, permissions, uid/gid ownership, and modification times.
The bit about "reverse diffs" comes in handy if backing up large files with small, frequent modifications. This led me to choose rdiff-backup
vs another similar and very good program I've used in the past called rsnapshot.
Whereas rdiff-backup
stores a diff file containing the changes a file undergoes, rsnapshot
creates a full copy of the modified file. The advantage in the full copy approach is the ease of restoring a file. Just navigate to the desired copy in the backup and cp
to a new location, whereas rdiff-backup
has to "re-assemble" the file from its collection of diffs.
The disadvantage kicks in when you're dealing with large files. Whereas rsnapshot
would create a full copy of the changed files on every run, rdiff-backup
only backs up a single full copy, then tracks the small daily changes that are made.
Setup
- Server is running Debian and desktop is running LMDE
- Both devices are on the same LAN
- Desktop
$HOME
is automatically synced to server and backed up daily
This is how I do it ...
1. SSH Keys
Disable password logins and switch to SSH key-based authentication to secure access and allow automated syncs from clients to remote servers. Read more
2. On the server: Sync directory
Create the sync
directory on the server to hold the contents of $HOME
that will be synced from the client ...
$ mkdir ~/sync
3. On the desktop: rsync + keychain
I use rsync
to sync the contents of my home directory to the server, and keychain
to supply the SSH key created in Step 1 to the script I create in Step 4. Cron daemon will send reports to the user when the task of backing up is later automated, which can be read in the terminal with the mail
command (provided by package bsd-mailx
)
Install ...
$ sudo apt install rsync keychain bsd-mailx
4. On the desktop: Sync script
A sample sync_home_to_server
script that syncs the contents of $HOME
to the sync
directory on the server ...
#!/usr/bin/env bash
set -euo pipefail
SOURCE_DIR="/home/<your_username>/"
DESTINATION_DIR="<ip_address_of_home_server>:${SOURCE_DIR}sync/"
EXCLUDE="--exclude **.cache --exclude **.part --exclude **.qcow2 --exclude **Trash"
OPTS="--archive --verbose --delete"
# Make available the unlocked SSH key to the script when its run from a cron job:
. ${SOURCE_DIR}.keychain/<name_of_key>-sh
rsync $EXCLUDE $OPTS $SOURCE_DIR $DESTINATION_DIR
Run the first sync (subsequent syncs will be much faster) ...
$ ./sync_home_to_server
5. On the server: Backup directory
Install ...
$ sudo apt install rdiff-backup bsd-mailx
Directory ~/sync
now holds a mirror copy of the contents of $HOME
on the client as it existed at the time of the last sync operation.
Create /home/backup
to serve as the backup directory ...
$ sudo mkdir /home/backup
$ sudo chown <your_username>: /home/backup
6. On the server: First backup
Run rdiff-backup
to make the first backup (subsequent backups will be much faster) ...
$ rdiff-backup backup /home/<your_username>/sync/ /home/backup/
When the process is complete, /home/backup
will be a mirror copy of /home/<your_username>/sync
with one exception: a new /home/backup/rdiff-backup-data
directory is created to store the logs, metadata, and incremental changes.
View backup session statistics by running rdiff-backup
with the --print-statistics
option, or afterwards by running ...
$ rdiff-backup-statistics /path/to/backup/directory/
7. On the server: Backup script
A sample backup_home
shell script that syncs the contents of ~/sync
to /home/backup
, increments the previous backups, and removes any increments more than 2 weeks old ...
#!/usr/bin/env bash
set -euo pipefail
SOURCE_DIR="/home/<your_username>/sync/"
DESTINATION_DIR="/home/backup/"
rdiff-backup backup --print-statistics $SOURCE_DIR $DESTINATION_DIR
rdiff-backup remove increments --older-than 2W $DESTINATION_DIR
Run ...
$ ./backup_home
8. On the server: Automate backups
Create a cron job to:
- Run
backup_home
to perform a daily backup at 01:10 - Email a job summary to
<your_username>
$ crontab -e
Sample job ...
10 1 * * * /path/to/backup_home
9. On the desktop: Automate syncs
Create a cron job to:
- Run
sync_home_to_server
to perform a daily sync at 12:10 - Email a job summary to
<your_username>
$ crontab -e
Sample job ...
10 12 * * * /path/to/sync_home_to_server
10. One more thing ...
Done! All the files in my home directory that I care about are synced to the server, where they are incrementally backed up every day. Don't have to think about it!
But when I do think about backups, I have a few LUKS-encrypted external USB storage devices that I rotate into service and perform manual backups to using rsync
. I also recommend storing at least one of the storage devices off-site, and periodically swap the drives.
» Next: Chromebook to Bookwormbook
« Previous: #25. Space CPU