A backup you don't have to think about is a backup that gets done

Last updated on 2022-09-18 Tagged under  #linux   #debian   #homeServer 

Backups are an easy task to put off but wow do you feel the pain when a file is mistakenly deleted or a hard drive fails!

I use syncthing to continuously sync files from my desktop home directory to the server, where a cron job is run daily that uses rdiff-backup to backup those synced files to a designated backup directory.

As per the rdiff-backup manpage ...

The target directory ends up a copy (mirror) of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves symlinks, special files, hardlinks, permissions, uid/gid ownership, and modification times.

The bit about "reverse diffs" comes in handy if backing up large files with small, frequent modifications. This led me to choose rdiff-backup vs another similar and very good program I've used in the past called rsnapshot.

Whereas rdiff-backup stores a diff file containing the changes a file undergoes, rsnapshot creates a full copy of the modified file. The advantage in the full copy approach is its easy to restore a file. Just navigate to the desired copy in the backup and cp to a new location, whereas rdiff-backup has to "re-assemble" the file from its collection of diffs.

The disadvantage kicks in when you're dealing with large files. I use thunderbird as my email client and some of its files are several GB in size and change every day; rsnapshot would create a full copy every day of the changed files, rdiff-backup only backs up a single full copy, then tracks the small daily changes that are made.

This is how I do it ...

1. Install

Install rdiff-backup ...

$ sudo pip3 install rdiff-backup

2. Backup

On my home server, syncthing has a mirror copy of my files in /home/<username> (replace <username> with your own).

I create /home/backup to serve as my backup directory ...

$ sudo mkdir /home/backup
$ sudo chown <username>: /home/backup

Run rdiff-backup to make the first backup ...

$ rdiff-backup /home/<username>/ /home/backup/

When the process is complete, /home/backup will be a mirror copy of /home/<username> with one exception: a new /home/backup/rdiff-backup-data directory is created to store the logs, metadata, and incremental changes.

3. Automate

I create a cron job (crontab -e) to:

15 1 * * * /usr/bin/rdiff-backup --exclude /home/<username>/.cache --print-statistics /home/<username>/ /home/backup/

4. One more thing ...

Done! All the files I care about are synced from devices to the server, where they are incrementally backed up every day. Don't have to think about it!

But when I do think about backups ... I have an external USB drive for manual backups. A simple rsync -av <source>/ <destination>/ takes a snapshot of my /home. I also have another USB drive stored off-site, and I periodically swap the drives.

Thanks for reading! Read other posts?

» Later: Hello 100 Days of Code

« Earlier: DoH and custom DNS servers with OpenWrt