Miskatonic University Press

I did some maintenance on my backup system this week, and for posterity I’ll document how it works, including how to set up an encrypted hard drive in Linux (in my case, Ubuntu).

Hard drive docks

I do backups to hard drives sitting in a dock, which I attach when needed with USB. The dock I have right now is made by Vantec and takes two hard drives. I got it, and the drives, at Canada Computers (which these days, by the way, has a great curbside pickup service).

These docks make it easy to have lots of cheap storage when needed. You can leave drives sitting in them or take them out or swap them around (carefully) as needed and then you just plug it in and there are your terabytes of disk. For backups, where speed doesn’t matter, you can buy slower and cheaper drives. The larger 3.5” drives are cheaper than 2.5” drives that go inside laptops, or, of course, solid state drives.

Backup scripts

There’s a primary backup drive, to which I copy everything, and a mirror, which is an exact copy of the primary. Every week I run my backup scripts to refresh everything on the main drive, and then I refresh the mirror. The primary right now is CRYPT_THREE, and backup scripts look like this one, which backs up my web site from its host:

#!/bin/sh -x

BACKUP_DRIVE=CRYPT_THREE

PAIRBACKUPS=/media/wtd/${BACKUP_DRIVE}/backups/pair

rsync -avz --rsh="ssh -q" --delete --times --progress pair:public_html/miskatonic.org/ "$PAIRBACKUPS/miskatonic.org/"

(Now that I look at it, those rsync options need tidying. --times is redundant, and I like to use GNU-style long option names, so it should be --archive --verbose --compress --delete --progress --rsh="ssh -q".) I don’t think I need the rsh option, though. I’ll fiddle with it. But it works.)

Backing up my laptop looks like this:

#!/bin/sh -x

BACKUP_DRIVE=CRYPT_THREE

dpkg --get-selections > ~/backups/marcus-packages.txt

dump-library.sh

rdiff-backup --verbosity 5         \
             --include /home/wtd/             \
             --include /var/www/              \
             --include /etc/apache2/          \
             --include /etc/hosts             \
             --exclude '**'                   \
             / /media/wtd/${BACKUP_DRIVE}/backups/marcus/

The dpkg command makes a list of all of the packages currently installed, and dump-library.sh does a dump of the database behind my personal library catalogue, Mrs. Abbott.

rdiff-backup does differential backups, taking snapshots of my files at that moment. I can go back and look at things as they were last month or a couple of years ago. All the other backups are just mirroring what’s there on remote machines, but for the laptop where I do everything this means if I need something I deleted years ago I can go back and find it. (Every now and then I wipe out a year’s worth with something like rdiff-backup --remove-older-than 2014-01-01 /media/wtd/CRYPT_THREE/backups/marcus/.)

And now that I look at this I see there’s more in /etc/ I should be backing up, so I’ll tweak that.

All those scripts get everything onto my primary backup drive, currently CRYPT_THREE. This mirrors it to CRYPT_TWO (and again --times is redundant):

#!/bin/sh

BACKUP_DRIVE=CRYPT_THREE
MIRROR_DRIVE=CRYPT_TWO

rsync --archive --verbose --delete --times --exclude "/lost+found/" /media/wtd/${BACKUP_DRIVE}/ /media/wtd/${MIRROR_DRIVE}/

Making an encrypted drive

Making those encrypted drives takes some special commands. Here’s how I did it when I bought a new 4 TB drive because my 2 TB backups were running out of space. (Don’t ask me how dm-crypt actually works.)

First, I took out the other drives from the hard drive dock. I put the new drive in and turned it on. It’s unformatted, so it didn’t get mounted, but the computer knew it was there and gparted saw it. It was identified as /dev/sdb. I set the partition table type to GPT (GUID Partition Table).

Always make sure you’re formatting and encrypting the new hard drive and not the drive in your laptop with all your files on it! That way madness lies. Scrutinize the /dev/sdb stuff very carefully. My main drive is /dev/sda. A, B, A, B … I always check multiple times before I do anything serious.

Then:

sudo cryptsetup luksFormat /dev/sdb

Here I confirmed I know what I’m doing and then entered the passphrase for the disk. I always type the passphrase in by hand (twice) to make sure nothing funny happens from copying and pasting, like a weird end-of-line character or a space sneaking in accidentally.

Then I put a file system on it and named it, in this case CRYPT_THREE.

sudo cryptsetup open /dev/sdb CRYPT_THREE
sudo mkfs.ext4 /dev/mapper/CRYPT_THREE
sudo cryptsetup close CRYPT_THREE
sudo cryptsetup --type luks open /dev/sdb CRYPT_THREE
sudo cryptsetup close CRYPT_THREE

I don’t know where I got that incantation, but it’s copied from somewhere. I don’t know if the close followed directly by an open could be collapsed, but it works and I’m not going to touch it. (On the other hand, there’s nothing to lose when you’re setting up a new hard drive because it’s empty and you can just reformat it, so maybe next time I’ll look into it.)

Now it was ready. I safely unmounted the drive and turned the dock off, then turned it on and checked what happened. The system saw the drive and asked for the passphrase, then mounted it. On Ubuntu, it shows up as /media/wtd/CRYPT_THREE. It works!

Moving drives around

Before I got the new 4 TB drive, I had two 2 TB drives. Call them A and B: A was the primary, B was the mirror. I wanted to make the new drive the primary, and keep the mirror. The steps were:

• Format the new drive, C.
• Mirror A to C.
• Change my scripts so that they back up to C.
• Run all my scripts to do fresh backups to C.
• Mirror C to B.
• Confirm everything works. (If it doesn’t, A is still good to be the primary.)

Now C is my primary backup and B still the mirror. That left me with A, which became my offsite backup. Normally, next I’d take B offsite and bring back A, then mirror C to A and make A my local mirror, and keep switching A and B regularly. However, by then I will have moved everything to 4 TB drives, so I’ll have D and E. I’ll set them up and mirror C to both of them, then use D as my local mirror and take E offsite. When I bring back A that will leave me with A and B as superfluous 2 TB drives. I have some ideas for how to use them, and will write them up if things work out.

Here is my offsite backup tucked into a special protective case before I took it somewhere (obeying all isolation directives) to sit on a quiet shelf.