A backup you don't have to think about is a backup that gets done

Last updated on 2021-06-18 Tagged under  # linux  # network

Part of "New life for an old laptop as a Linux home server"

Backups are an easy task to put off but wow do you feel the pain when a file is mistakenly deleted or a hard drive fails!

I use syncthing to continuously sync files from my desktop home directory to the server, where a cron job is run daily that uses rdiff-backup to backup those synced files to a designated backup directory.

As per the rdiff-backup manpage ...

The target directory ends up a copy (mirror) of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves symlinks, special files, hardlinks, permissions, uid/gid ownership, and modification times.

The bit about "reverse diffs" comes in handy if backing up large files with small, frequent modifications. This led me to choose rdiff-backup vs another similar and very good program I've used in the past called rsnapshot.

Whereas rdiff-backup stores a diff file containing the changes a file undergoes, rsnapshot creates a full copy of the modified file. The advantage in the full copy approach is its easy to restore a file. Just navigate to the desired copy in the backup and cp to a new location, whereas rdiff-backup has to "re-assemble" the file from its collection of diffs. The disadvantage kicks in when you're dealing with large files. I use thunderbird as my email client and some of its files are several GB in size and change every day; rsnapshot would create a full copy every day of the changed files, rdiff-backup only backs up a single full copy, then tracks the small daily changes that are made.

1. Install

On Debian 10 buster the rdiff-backup package is very old. Use pip3 install to grab the latest stable package.

First, install some dependencies ...

$ sudo apt install python3-pip python3-setuptools python3-pylibacl python3-pyxattr

Install rdiff-backup ...

$ sudo pip3 install rdiff-backup

2. Backup

On my home server, syncthing has a mirror copy of my files in /home/<username> (replace <username> with your own). I create /home/backup to serve as my backup directory ...

$ sudo mkdir /home/backup
$ sudo chown <username>: /home/backup

Run rdiff-backup to make the first backup ...

$ rdiff-backup /home/<username>/ /home/backup/

When the process is complete, /home/backup will be a mirror copy of /home/<username> with one exception: a new /home/backup/rdiff-backup-data directory is created to store the logs, metadata, and incremental changes.

3. Automate

I create a cron job (crontab -e) to run a daily backup in the early morning hours ...

10 1 * * * /usr/local/bin/rdiff-backup --exclude /home/<username>/.cache --print-statistics /home/<username>/ /home/backup/

4. One more thing ...

Done! All the files I care about are synced from devices to the server, where they are incrementally backed up every day. Don't have to think about it!

But ... When I do think about backups I have an external USB drive that I use periodically for manual backups. A simple rsync -av <source>/ <destination>/ that takes a snapshot of my /home. Then I have a second USB drive stored off-site, and I swap the drives.

» Later: Nginx web server

« Earlier: Sync files across all computers using Syncthing