About three years ago I wrote a post on how to setup duplicity and by that time I was using the program for about two years. Not long after I made that post I realized that it wasn't the best backup solution for me. The two biggest reasons why I ditched duplicity was first, it was slow and second it ate up a lot of disk space. The two reasons go hand-in-hand. Duplicity doesn't do any deduplication, which means if I move a directory that is 100GB, and there was only 1GB of changes to existing files, the entire backup for the day would be 101GB.
For a single computer that may not be that big of a deal, providing a lot of files aren't being moved around. Since I do web development I have a lot of vendor and node_module directories. If I have 75 projects and each one has a node_modules and vendor that are 500MB and 100MB, respectively, then duplicity would need 37.5GB for all the node_module directories and 7.5GB for all vendor directories. That's a lot of wasted write operations that puts unnecessary strain on the drives. At the time I was running 12 x 1TB drives in RAID6 and in those two years I went through 4 internal drives and 2 external drives. I will admit that I was using normal desktop drives such as WD Blue but even if I was running WD RE or Gold drives, they'd still fail a lot sooner than normal as I was pushing at least 100GB a night to them.
I looked at other solutions and had implemented bacula but as great as that was it was a bit overkill for my needs. Then I found borgbackup which is simple, offers deduplication, and encryption.
Setup
I have a system with 28TB of disk space which will be used for backups. There are two ways you can approach this. The first is mounting the borg repository over the network. The second way is creating a backup user and using ssh. The latter requires that borg be installed on both the server and the local machines.
Backup Server
Create the Backup User
If you're going to go the route that I took where I have a user account specifically for backups then the first thing to do is create the user account:
mkdir /srv/backup
groupadd backup
useradd -m -d /home/backup -s /bin/bash -g backup backup
chown -R backup:backup /srv/backup
Install Borg (Option One)
Borg will need to be installed on the server if your are not going to network-mount the repository directory on the system that is to be backed up.
| Distribution | Source | Command |
|---|---|---|
| Debian-based | various | apt install borgbackup |
| Gentoo | ebuild | emerge borgbackup |
| MacOS | Homebrew | brew install borgbackup |
For more distributions see Borg Installation.
Generate a new SSH key for backup use only.
sudo su
ssh-keygen -t rsa -f ~/.ssh/backup_id_rsa
ssh-copy-id -i backup_id_rsa backup@backup-server
Network File System (Option two)
I went with the SSH route and opted to not use any network file systems. Therefore setting up network shares is beyond the scope of this because once configured properly, it's no different than a local borg repository.
Create Repository
There are three options for encrypting the repository and they are none, passphrase-only, and passphrase and key. Obviously none means the repository won't be encrypted which can speed things up if security isn't a concern. The other two provide encryption and have keys. The only difference between the two is where the key is stored. In passphrase-only the key is stored within the repository and you need to provide a passphrase to decrypt the key in order to decrypt the rest of the repository. The passphrase and key stores the key outside of the repository.
# Local repository, repokey encryption, BLAKE2b (often faster, since Borg 1.1)
borg init --encryption=repokey-blake2 /srv/backup
# Local repository (no encryption)
borg init --encryption=none /srv/backup
# Remote repository (accesses a remote borg via ssh)
# repokey: stores the (encrypted) key into /config
borg init --encryption=repokey-blake2 backup@backup-server:/srv/backup
# Remote repository (accesses a remote borg via ssh)
# keyfile: stores the (encrypted) key into ~/.config/borg/keys/
borg init --encryption=keyfile backup@backup-server:/srv/backup
If you ever forget which encryption type you used during initialization, you can find out by first looking in the /path/to/repo/config file. Get the ID from that file and then look in the key-type file in ~/.config/borg/security/$ID. There will be a number in that file.
- 0: keyfile
- 1: Not used
- 2: none
- 3: repokey
- 4: keyfile-blake2
- 5: repokey-blake2
- 6: authenticated-blake2
- 7: authenticated
System(s)
Borg will need to be installed on each system that you want to backup data from.
When backups are created, there is a cache that is stored within the ~/.cache/borg directory. However, in my case this is located on a SSD array with limited space. Therefore I have a directory called /home/backup that resides on my spinning disk array. This ensures that I don't wear out my SSDs and have ample disk space for backup caches.
If /root/.cache/borg works for you, then you can leave it as is. However, if you need to move the cache to a different location then you can set the BORG_BASE_DIR.
Creating Backups
For consistency, it's best to create a script and run that for both your automated and manual backups.
The general command that I use when I make backups, regardless if they are remote or local is:
borg create --stats --list --verbose --filter=AME --exclude-caches --exclude-nodump \
--one-file-system --chunker-params 16,23,31,4095 --comment "$(uname -a)" \
--exclude '/home/lost+found' \
--exclude '/home/backup' \
--exclude '/home/*/.cache' \
--exclude '/home/*/.local/share/Trash' \
--exclude '/home/*/.vagrant.d/boxes/*' \
--exclude '/home/*/Downloads' \
--exclude '*/node_modules' \
--exclude '*/vendor' \
::'{hostname}-{now:%Y-%m-%d}' /etc /home /root
Your exclude patterns may differ but at the very least, you should exclude /home/backup if that's where you are storing your borg cache.
Automation
I saved the following file in /etc/cron.daily/backup. One thing to note is that for automatic backups to work if there is a password, then you will need to let borg know what that password is. This can be done a number of ways. You can use the BORG_PASSPHRASE, BORG_PASSCOMMAND, or BORG_PASSPHRASE_FD. I prefer the BORG_PASSCOMAND since it's more flexible. For example, if you have a Yubikey or OnlyKey, you can set something up to get the password from that rather than cat /path/to/password-file.
For the sake of simplicity, let's just store the password in a regular file.
echo "super-secret-password" > /root/.borg-passphrase
Modify the exports to fit your needs as well as the excludes.
#!/bin/bash
export BORG_RSH="ssh -i /root/.ssh/backup_id_rsa"
export BORG_REPO=ssh://backup@backup-server:22/srv/backup
export BORG_PASSCOMMAND="cat /root/.borg-passphrase"
export BORG_BASE_DIR="/home/backup"
borg create --stats --list --verbose --filter=AME --exclude-caches --exclude-nodump \
--one-file-system --chunker-params 16,23,31,4095 --comment "$(uname -a)" \
--exclude '/home/lost+found' \
--exclude '/home/backup' \
--exclude '/home/*/.cache' \
--exclude '/home/*/.local/share/Trash' \
--exclude '/home/*/.vagrant.d/boxes/*' \
--exclude '/home/*/Downloads' \
--exclude '*/node_modules' \
--exclude '*/vendor' \
::'{hostname}-{now:%Y-%m-%d}' /etc /home /root
borg prune -n --list --prefix '{hostname}' --keep-within 1y ::
The last line, the borg prune --list --prefix '{hostname}' --keep-within 1y :: can also be adjusted to your needs or completely removed.
Manual Backups
Generally, it should be rare that you need to create a manual backup. However, from time to time you may need or want to create a "save point" if you're moving or deleting a lot of files.
If you created a cronjob by adding a file into the /etc/cron.daily/backup then you could simply run that file whenever you need to.
Maintenance
Check
The check command verifies the consistency of a repository and the corresponding archives. Using the --repair may cause data loss, so use with caution.
borg check --verify-data ::
Rename
Simply renames an archive:
borg rename ::old-name new-name
List
List all archives in a repository:
borg list ::
List all files within an archive:
borg list ::archive
For listing files in a specific format, see list description
Diff
The diff sub-command simply finds the difference in archives within the repository.
borg diff ::archive1 archive2
Delete
Deletes an archive within the repository.
borg delete ::archive
Prune
The prune sub-command is the delete sub-command but on steroids. It allows you to specify what archives you want to delete and keep.
borg prune --keep-within 2m ::
borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 ::
See the Prune Page for more information.
Info
Displays information on the archive or repository.
borg info ::
borg info ::archive
break-lock
Does what the command says, breaks the repository's lock. This should be used with caution.
borg break-lock ::
Recovery
Extract
Extracts the contents or specified paths. Paths can be further limited by using the --exclude option. Using the --stdout can be used to restore drive images.
borg extract ::2021-02-21
borg extract ::2021-02-21 home
borg extract ::2021-02-21 home --exclude '.Trash'
borg extract ::external-drive | dd of=/dev/sdc bs=10M
mount/unmount
Allows you to mount an archive to browse the archive as if it were an actual file system. This sub-command uses the FUSE filesystem. The unmount sub-command does what it says, unmount a borg repository.
borg mount ::2021-02-21 /mnt/borg
borg unmount /mnt/borg
export-tar
Essentially the same as the extract sub-command but creates a tar file instead of restoring the directory structure and each individual file within those directories. Use the --tar-filter=
borg export-tar ::2021-02-21 Archive.tar borg export-tar --tar-filter=lrzip -L 9" ::2021-02-21 Archive.tar.lrz borg export-tar ::2021-02-21 - | curl --data-binary @- https://server.net/endpoint borg export-tar ::2021-02-21 - | ssh www "cd /var/www/html ; tar x"