Home→Archive↴

A Backup You Don't Have to Think About is a Backup that Gets Done

Last edited on 2023-08-08 • Tagged under  #server   #linux 

Backups are an easy task to put off but WOW do you feel the pain when a file is mistakenly deleted or a storage device fails!

I use a combination of rsync + ssh + rdiff-backup + cron to perform an automated daily backup of my desktop home directory to my home server.

As per the rdiff-backup manpage:

The target directory ends up a copy (mirror) of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves symlinks, special files, hardlinks, permissions, uid/gid ownership, and modification times.

The bit about "reverse diffs" comes in handy if backing up large files with small, frequent modifications. This led me to choose rdiff-backup vs another similar and very good program I've used in the past called rsnapshot.

Whereas rdiff-backup stores a diff file containing the changes a file undergoes, rsnapshot creates a full copy of the modified file. The advantage in the full copy approach is the ease of restoring a file. Just navigate to the desired copy in the backup and cp to a new location, whereas rdiff-backup has to "re-assemble" the file from its collection of diffs.

The disadvantage kicks in when you're dealing with large files. Whereas rsnapshot would create a full copy of the changed files on every run, rdiff-backup only backs up a single full copy, then tracks the small daily changes that are made.

So... rdiff-backup it is!

This is how I do it...

1. Setup

  • SERVER is running Debian and DESKTOP is running LMDE
  • Both devices are on the same LAN
  • DESKTOP $HOME is automatically synced to SERVER and backed up daily

2. On the SERVER and the DESKTOP: Secure access using SSH keys

Configure remote access to the target device using SSH keys, and disable password logins. Read more

3. On the SERVER: Create sync directory

Create the sync directory on the SERVER to hold the contents of $HOME that will be synced from the DESKTOP:

mkdir ~/sync

4. On the DESKTOP: Install tools

sudo apt install rsync keychain bsd-mailx

I use rsync to sync the contents of my home directory to the server, and keychain to supply the SSH key created in Step 2 to the script I create in the next step.

Cron daemon will send reports to the user when the task of backing up is later automated, which can be read in the terminal with the mail command (provided by package bsd-mailx).

5. On the DESKTOP: Create sync script

A sync_home_to_server example script that syncs the contents of $HOME to the sync directory on the server:

#!/usr/bin/env bash

set -euo pipefail

EXCLUDE="--exclude **.cache --exclude **Trash"
OPTIONS="--archive --verbose --delete"
SERVER="<ip_address_of_SERVER>:${HOME}/sync/"
KEY="${HOME}/.keychain/$(hostname)-sh"

# Make available the unlocked SSH key to the script when its run from a cron job:
. $KEY

rsync $EXCLUDE $OPTIONS ${HOME}/ $SERVER

Run the first sync (subsequent syncs will be much faster):

./sync_home_to_server

6. On the SERVER: Install tools

sudo apt install rdiff-backup bsd-mailx

7. On the SERVER: Create backup directory

Directory ~/sync now holds a mirror copy of the contents of $HOME on the client as it existed at the time of the last sync operation.

Create /home/backup to serve as the backup directory:

sudo mkdir /home/backup
sudo chown <your_username>: /home/backup

8. On the SERVER: First backup

Run rdiff-backup to make the first backup (subsequent backups will be much faster):

rdiff-backup backup ${HOME}/sync/ /home/backup/

When the process is complete, /home/backup will be a mirror copy of ${HOME}/sync with one exception: a new /home/backup/rdiff-backup-data directory is created to store the logs, metadata, and incremental changes.

View backup session statistics by running rdiff-backup with the --print-statistics option, or afterwards by running:

rdiff-backup-statistics /home/backup/

9. On the SERVER: Create backup script

A backup_home example script that syncs the contents of ~/sync to /home/backup, increments the previous backups, and removes any increments older than two weeks:

#!/usr/bin/env bash

set -euo pipefail

DESTINATION="/home/backup/"

rdiff-backup backup --print-statistics ${HOME}/sync/ $DESTINATION
rdiff-backup remove increments --older-than 2W $DESTINATION

Run:

./backup_home

10. On the SERVER: Automate backups

Create a cron job that:

  • Runs backup_home to perform a daily backup (example: scheduled for 01:10)
  • Emails a job summary to <your_username>
crontab -e

Example:

10 1 * * * /path/to/backup_home

11. On the DESKTOP: Automate syncs

Create a cron job that:

  • Runs sync_home_to_server to perform a daily sync (example: scheduled for 12:10)
  • Email a job summary to <your_username>
crontab -e

Sample job:

10 12 * * * /path/to/sync_home_to_server

12. One more thing...

Done! All the files in my home directory that I care about are synced to the server, where they are incrementally backed up every day. Don't have to think about it!

But when I do think about backups, I have a few encrypted USB hard drives that I rotate into service and perform manual backups to using rsync. I also store at least one of the hard drives off-site, and periodically swap the drives.

Thanks for reading! Read other posts?

» Next: Chromebook to Bookwormbook

« Previous: Remotely Unlock an Encrypted Linux Server Using Dropbear