I am running btrfs for quite some time now and I have created a simple script to backup to a second hard-drive via btfrs send.
I recently started to look into snapper and like what I see so far. However, I am not sure how I go about using a snapper created snapshot for my differential backup.
With my existing script a backup looks like this (simplified):
btrfs send -p /mySnapshots/2020-10-01 /mySnapshots/2020-10-02 | btrfs receive /pathToBackupDestination
After this command I will end up with a new folder reflecting todays delta backup:
/pathToBackupDestination/2020-10-01
/pathToBackupDestination/2020-10-02
But with snapper all snapshots have the same name and the source snapshots are only separated by their location. Which means btrfs can’t create the delta in the destination.
First run, create the initial snapshots, no delta possible
btrfs send /.snapshots/335/snapshot | btrfs receive /pathToBackupDestination
Result is:
/pathToBackupDestination/snapshot
When I now run this:
btrfs send -p /.snapshots/335/snapshot /.snapshots/336/snapshot | btrfs receive /pathToBackupDestination
things wail fail because btrfs will try to overwrite “snapshot” which it needs to build the delta
Means I can only overwrite an existing /pathToBackupDestination/snapshot but can’t create the delta.
I can obviously rename the source “snapshot” to something unique but that will likely break snapper.
I hope I am making sense?
Anyone came across this already?
I had a quick look at snap-sync, btrfs-backup and btrfs-sxbackup and it seems they all make and handle their own set of snapshots. One would have to fiddle to get them to use preexisting timeshift or snapper snapshots as they don’t allow this out of the box, as far as I could see at first glance. So a quick renaming seems preferable .
There are some prebuilt tools for replicating snapper snapshots. Have you looked at how they solve the problem? Alternatively, you could just use one of them. Here is an example:
Very good advice! And I think I have just learned about a new parameter
-c instead of -p
# Sends the difference between the new snapshot and old snapshot to the
# backup location. Using the -c flag instead of -p tells it that there
# is an identical subvolume to the old snapshot at the receiving
# location where it can get its data. This helps speed up the transfer.
Edit:
Unfortunately this parameter doesn’t overcome the naming issue, but looking into this in more detail I realised a flaw in my approach, and that is that whenever I start a backup Snapper will have created quite a few snapshots in the meantime, which means I somehow need to capture what snapshot I used for my last delta backup. Not impossible but new moving parts which complicate things.
Anyway, this new parameter is interesting and may speed up my existing backup routine as well.
Removed snapper again, too much “behind the scenes” stuff going on because it needs to be flexible which I didn’t want to bother investigating.
So I wrote my own little snapper.sh which does exactly what I want and which stores stuff exactly where I want
Highly simplistic in comparison.
With the current settings the script will keep 10 copies of @ and @home, all stored in /mnt/snapshots.
I plan to run it manually whenever I feel like it, e.g. before a pacman run, and created a systemd timer which will run it every hour.
version="1.0.1"
Divider1="##############################################################################"
Divider2="------------------------------------------------------------------------------"
# Define enhanced echo function
function print {
tmp=${1:0:2}
case "$tmp" in
"E:") echo -e "\e[0;31m"$(date +%T) - $1 "\033[0m";;
"W:") echo -e "\e[1;33m"$(date +%T) - $1 "\033[0m";;
"##") echo "$1";;
'- ') echo "$1";;
*) echo -e $(date +%T) - "$1"
esac
}
# $1 = Snapshot directory
# $2 = Name tag
# $3 = Snapshots to keep
function deleteSnapshots {
list=$(ls -rd $1/$2* | awk 'NR>'$3)
if [ "$list" == "" ]
then
print "I: List is empty, nothing to do"
else
btrfs subvolume delete $list
fi
}
# $1 = Source directory
# $2 = Target directory
# $3 = Snapshot tag
function createSnapshot {
ts="$3$(date +%Y-%m-%d-%H:%M)"
target="$2/$ts"
# echo $target
btrfs subvolume snapshot $1 $target
}
if [ "$EUID" -ne 0 ]
then print "E: Script needs to run as root!"
exit 1
fi
count=10
tag="Hourly_@_"
echo $Divider1
createSnapshot "/mnt/@" "/mnt/snapshots" $tag
echo $Divider2
deleteSnapshots "/mnt/snapshots" $tag $count
echo $Divider2
tag="Hourly_@home_"
createSnapshot "/mnt/@home" "/mnt/snapshots" $tag
echo $Divider2
deleteSnapshots "/mnt/snapshots" $tag $count
echo $Divider1
#
# Important! The below will only work with Systemd-Boot, not with Grub or ReFind
#
print "Update boot options with latest snapshot…"
cd /mnt/snapshots
snapshot=$(ls -dtr *@_* | head -1)
defaultConf=$(cat /boot/loader/loader.conf | grep default | cut -d" " -f2)
print "Default config: $defaultConf"
print "Most recent snapshot: $snapshot"
newConfig=$(echo "/boot/loader/entries/$snapshot.conf" | sed '1s/:/_/')
print "Create new boot entry: $newConfig"
cp /boot/loader/entries/$defaultConf $newConfig
sed -i "s/@/snapshots\/${snapshot}/" $newConfig
echo $Divider2
# Delete
print "Delete oldest boot snapshot"
cd /boot/loader/entries
deleteConfig=$(ls -atr *@_* | head -1)
print "Delete oldest boot entry: $deleteConfig"
rm $deleteConfig
echo $Divider1
print Done!
Edit: Inspired by @2000 btrfs thread I added a few more lines to my script to add the last x snapshots to Systemd-Boot so one can select the latest snapshot when booting. Not perfect nor bullet proof but maybe of help for someone else.
.