aboutblognotesprojects

More posts

Server back-ups with Restic

27 November 2021 (7 months ago)🏷️ #100daystooffload🏷️ #technology🏷️ #selfhost

💯 100 Days to Offload

This article is one of a series of posts I have written for the 100 Days to Offload challenge . Disclaimer: The challenge focuses on writing frequency rather than quality, and so posts may not always be fully planned out!

View other articles in this series

A while ago I posted about how I back-up my personal servers to Backblaze B2. That approach involved adding all of the files and directories into a single compressed archive and sending this up to an encrypted bucket on B2.

Whilst this does achieve my back-up goals (and I use Telegram to notify me each time it completes), it felt inelegant. Every time the back-up executed, the entirety of the back-up file - several gigabytes - would be built and transferred.

In this post I will talk about my new approach, which uses a tool called Restic.

Restic, repositories, and Backblaze B2

Restic is an open-source cross-platform program to back-up files and directories. It works on the basis of snapshots and repositories.

A single repository can contain multiple "snapshots" representing various back-ups. Each snapshot is timestamped and can be filtered by the path of the back-up and the host that the back-up came from.

One of the most useful features of Restic is that each snapshot only stores the representation of what changed since the last snapshot. This means that although the first back-up might take a little while, subsequent snapshots are much quicker.

Repositories can be local (i.e. on a filesystem accessible by the host), or remote. Restic supports a number of remote repository types - from SFTP and RESTful interfaces through to S3-compatible backends and storage services operated by cloud providers.

Restic also supports Backblaze B2 out of the box, which is the repository type that I use.

In the rest of this post I will run through how this can be set-up and automated.

Preparing a Restic repository on B2

First, we need to install Restic. There are instructions for installation on a range of systems on the Restic website.

Next, in the Backblaze B2 console, create a new bucket. The bucket will need a unique name. Whilst there, create a new "App Key", with read/write access to the bucket. Make a note of the keyID and applicationKey once created.

You can close the B2 console. You now need a file that can be sourced to tell Restic how to access your repository. Create a new file (e.g. in the home directory of the host you want to back-up), called resticenv, with these contents:

export B2_ACCOUNT_ID="<KEY_ID>"
export B2_ACCOUNT_KEY="<APPLICATION_KEY>"
export RESTIC_REPOSITORY="b2:<BUCKET_NAME>"
export RESTIC_PASSWORD="<PASSWORD>"

You'll need to replace some of the parts of the file as described below:

  • <KEY_ID>: the keyID of the B2 key you created earlier
  • <APPLICATION_KEY>: the applicationKey of the B2 key you created earlier
  • <BUCKET_NAME>: the name of the B2 bucket you created earlier
  • <PASSWORD>: a strong and private password

Each Restic repository is secured with a password. This is used to encrypt the repository, so if this is lost then so is the associated data. I use my password manager to generate a strong password. It goes without saying that this resticenv file should be kept very safe.

Finally, we need to initialise the repository. To do so, first source the environment file, and then use the init command:

$ source ~/resticenv
$ restic init

You're now ready to create back-ups.

Creating a new back-up

Creating a new back-up is as easy as running the following (as long as your shell still has the sourced environment as described above):

$ restic backup /path/to/directory

If the target directory is particularly large then the first back-up may take a while. However, subsequent back-ups will be much quicker.

Once done, you can run the following to view a list of all the snapshots held by the repository. You'll see your recently-created back-up listed:

$ restic snapshots

ID        Time                 Host        Tags      Paths
-----------------------------------------------------------------------
6b25863c  2021-11-27 08:39:44  myhost                /path/to/directory
-----------------------------------------------------------------------
1 snapshot

The next time you run restic backup again, you'll notice new snapshots are created.

Pruning snapshots

In some scenarios you may not want to keep back-ups around forever. For example, if you provide a service to users you may be in breach of your own Privacy Policy if you keep backed-up data of old deleted accounts.

Luckily it's straight forward to "forget" old snapshots. The command below tells Restic to hold onto the last 24 hours' worth of hourly snapshots and the last 7 days' worth of daily snapshots:

$ restic forget --prune --keep-hourly 24 --keep-daily 7

Recovering snapshots

Back-ups are useless unless you can later recover them! Rustic makes this easy. You just need the ID of the back-up you want to recover (as provided by the restic snapshots command from earlier):

$ restic restore 6b25863c --target ~/restored

The ~/restored directory should now contain the files included in the back-up at that snapshot's point in time.

Automating back-ups

Finally we can cover a way to automate back-ups. For this we can simply use cron. To begin, install a suitable cron service (if your system doesn't come with one). I use cronie.

Once ready, add an entry to your user's crontab by running crontab -e and adding the following line:

15 * * * * . /home/<USER>/resticenv; /usr/bin/restic backup -q /path/to/directory

The above job will run at quarter-past the hour for every hour the system is running. Note that cron requires that we include full paths to files. The -q flag suppresses output. You'll need to change parts of the line to suit your needs (run which restic to see where Restic is installed on your system).

Conclusion

I have much more confidence in a tool like Restic to handle my back-ups than a home-grown solution. It's great to have the peace of mind!

📲 Enjoyed this article? Subscribe to updates!

If you would like to read more posts like this, then you can subscribe via RSS.

Subscribe to an RSS feed