My backup regime

I wanted a regime that would do an incremental backup but also give me a complete image of my files at the time of the backup. Most backups either store multiple complete copies, which uses up disk space to quickly in my case, or do deltas of a snapshot taken some time in the past, which can make restoring files a real nightmare.

My solution is to do a reverse delta. I have a snapshot of the current state, which I update with any changes every night. However I also create a delta directory (named by day of backup) into which I copy any files that are about to be overwritten by updated versions.

This means that if I need to do a full restore. The files are there waiting for me. However, if for some reason I need to reach back in time, with a little fiddling, I can do so, and with a suitable program (not yet written) I can do so.

I am on Linux, so I have access to the excellent rsync utility, which will do exactly what I want. I could even use it to maintain a remote backup on a distant machine using ssh if I wanted (and perhaps should).

The script is shown below, but first a couple of downsides to this technique:

  1. Frequent even small changes to large files will still eat up space quickly
  2. You cannot delete intermediate deltas and still be able to reach back all the way to the first.

There are a lot of terms in here. At the bottom of the page, each is described (quotes from the manual).

#!/bin/bash

#Where to keep a log file of the backup
logfile='/home/chris/backuplogs/' #trailing slash important

#What to back up - you need to be specific here because this 
#will be run as root (scary)
backupfrom='/home/ myname /home/Shared /var'

#Where to keep the backups
backupto='/media/myname/vol2/rsyncbackups'
backupbackups='/media/myname/vol2/rsyncbackupdeltas'

#Where to log progress to
logfile='/home/myname/backup.log'

#datestamp for log files and to name the backup directory
datenow=$(date '+%F_%T')

#how many days of deltas to keep
keep=30

#Clear out old deltas so as to avoid running out of space  (this is 1 line)
find /media/myname/vol2/rsyncbackupdeltas/* -type d -mtime +${keep} | xargs rm -rf

#run the backup (this is 1 line)
sudo rsync  --archive  --verbose --atimes --delete --acls --owner --log-file=${logfile}rsync_${datenow}.log --backup-dir=${backupbackups}/on_${datenow} ${backupfrom} ${backupto} >${logfile}

Some notes on the options used in the script(alphabetical order)

–archive -a – is shorthand for -rlptgoD These are listed here & marked * below (recursive links perms times groups owner devices specials)

  • –acls -A – copy access control lists (how is this different from –perms??)
  • –atimes -U – copy last accessed times
  • –backup-dir= – for copies of destination files before they are overwritten/deleted.
  • –crtimes -N – copy creation times (omitted – not supported)
  • –delete – remove from target if file no longer in source
  • –devices -D – *copy data to allow device entries to be recreated
  • –dry-run -n – perform a trial run with no changes made
  • –exclude – the stuff you dont want to copy
  • –files-from= – read the list of files to backup from this file (causes knock on effects)
  • –group -g – *copy group info (only if source & dest are on same system)
  • –links -l – *recreate symlinks on the destination.
  • –log-file= – where to save all that verbosity
  • –omit-dir-times -O – *arcane but seems to be desirable
  • –owner -o – copy the owner (only if source & dest are on same system)
  • –perms -p – *copy file permissions
  • –progress – report progress (omitted – only relevant on slow conns.)
  • –recursive-r – *copy sub-folders too
  • –times -t – *copy modification times
  • –specials -D – *copy special files (or rather their spec)
  • –verbose -v – be chatty
  • –xattrs -X – copy source extended attrs to destination files
  • (–vverbose – logorrhea)

Note -D implies both –devices and –specials