A backup you don't have to think about is a backup that gets done

Last edited on 2023-08-08 Tagged under  #homeServer   #linux   #debian   #lmde 

Backups are an easy task to put off but WOW do you feel the pain when a file is mistakenly deleted or a storage device fails!

I use a combination of rsync + ssh + rdiff-backup + cron to perform an automated daily backup of my desktop home directory to my home server.

As per the rdiff-backup manpage ...

The target directory ends up a copy (mirror) of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves symlinks, special files, hardlinks, permissions, uid/gid ownership, and modification times.

The bit about "reverse diffs" comes in handy if backing up large files with small, frequent modifications. This led me to choose rdiff-backup vs another similar and very good program I've used in the past called rsnapshot.

Whereas rdiff-backup stores a diff file containing the changes a file undergoes, rsnapshot creates a full copy of the modified file. The advantage in the full copy approach is the ease of restoring a file. Just navigate to the desired copy in the backup and cp to a new location, whereas rdiff-backup has to "re-assemble" the file from its collection of diffs.

The disadvantage kicks in when you're dealing with large files. Whereas rsnapshot would create a full copy of the changed files on every run, rdiff-backup only backs up a single full copy, then tracks the small daily changes that are made.

Setup

  • Server is running Debian and desktop is running LMDE
  • Both devices are on the same LAN
  • Desktop $HOME is automatically synced to server and backed up daily

This is how I do it ...

1. SSH Keys

Disable password logins and switch to SSH key-based authentication to secure access and allow automated syncs from clients to remote servers. Read more

2. On the server: Sync directory

Create the sync directory on the server to hold the contents of $HOME that will be synced from the client ...

$ mkdir ~/sync

3. On the desktop: rsync + keychain

I use rsync to sync the contents of my home directory to the server, and keychain to supply the SSH key created in Step 1 to the script I create in Step 4. Cron daemon will send reports to the user when the task of backing up is later automated, which can be read in the terminal with the mail command (provided by package bsd-mailx)

Install ...

$ sudo apt install rsync keychain bsd-mailx

4. On the desktop: Sync script

A sample sync_home_to_server script that syncs the contents of $HOME to the sync directory on the server ...

#!/usr/bin/env bash

set -euo pipefail

SOURCE_DIR="/home/<your_username>/"
DESTINATION_DIR="<ip_address_of_home_server>:${SOURCE_DIR}sync/"
EXCLUDE="--exclude **.cache --exclude **.part --exclude **.qcow2 --exclude **Trash"
OPTS="--archive --verbose --delete"

# Make available the unlocked SSH key to the script when its run from a cron job:
. ${SOURCE_DIR}.keychain/<name_of_key>-sh

rsync $EXCLUDE $OPTS $SOURCE_DIR $DESTINATION_DIR

Run the first sync (subsequent syncs will be much faster) ...

$ ./sync_home_to_server

5. On the server: Backup directory

Install ...

$ sudo apt install rdiff-backup bsd-mailx

Directory ~/sync now holds a mirror copy of the contents of $HOME on the client as it existed at the time of the last sync operation.

Create /home/backup to serve as the backup directory ...

$ sudo mkdir /home/backup
$ sudo chown <your_username>: /home/backup

6. On the server: First backup

Run rdiff-backup to make the first backup (subsequent backups will be much faster) ...

$ rdiff-backup backup /home/<your_username>/sync/ /home/backup/

When the process is complete, /home/backup will be a mirror copy of /home/<your_username>/sync with one exception: a new /home/backup/rdiff-backup-data directory is created to store the logs, metadata, and incremental changes.

View backup session statistics by running rdiff-backup with the --print-statistics option, or afterwards by running ...

$ rdiff-backup-statistics /path/to/backup/directory/

7. On the server: Backup script

A sample backup_home shell script that syncs the contents of ~/sync to /home/backup, increments the previous backups, and removes any increments more than 2 weeks old ...

#!/usr/bin/env bash

set -euo pipefail

SOURCE_DIR="/home/<your_username>/sync/"
DESTINATION_DIR="/home/backup/"

rdiff-backup backup --print-statistics $SOURCE_DIR $DESTINATION_DIR
rdiff-backup remove increments --older-than 2W $DESTINATION_DIR

Run ...

$ ./backup_home

8. On the server: Automate backups

Create a cron job to:

  • Run backup_home to perform a daily backup at 01:10
  • Email a job summary to <your_username>
$ crontab -e

Sample job ...

10 1 * * * /path/to/backup_home

9. On the desktop: Automate syncs

Create a cron job to:

  • Run sync_home_to_server to perform a daily sync at 12:10
  • Email a job summary to <your_username>
$ crontab -e

Sample job ...

10 12 * * * /path/to/sync_home_to_server

10. One more thing ...

Done! All the files in my home directory that I care about are synced to the server, where they are incrementally backed up every day. Don't have to think about it!

But when I do think about backups, I have a few LUKS-encrypted external USB storage devices that I rotate into service and perform manual backups to using rsync. I also recommend storing at least one of the storage devices off-site, and periodically swap the drives.

Thanks for reading! Read other posts?

» Next: Chromebook to Bookwormbook

« Previous: #25. Space CPU