Some questions about Btrbk backup options and restoring

General system Applications
1

Hello,
I’d like to ask a few questions about Btrfs features and the btrbk tool in particular.

While browsing the forum, I found relatively few questions related to restoring backups using Btrfs features or frontend tools like btrbk intended to do so. Maybe except in this long post, which I haven’t finished reading.

A few users, including @dalto, seem to have mastered it very well, so I hope he’ll read my prose. End of parenthesis :slight_smile:

I want to use Btrbk.

Here’s my configuration: a triple-boot installation with systemd-boot that took me a long time to concoct (especially for distros that don’t offer a native Btrfs installer).

I don’t want to have to reinstall everything if my disk fails, adapt fstab etc.
A system backup a week or two old wouldn’t be too much trouble, but I’d like to keep manually installed apps as much as possible. I don’t have an insane amount of storage space either. At best the same capacity as the source disk.

I thought of using Btrbk for the attractive aspect of incremental backup, but I have a few reservations:

  • if the backup is fast, restoration is less so…
    → you have to restore the full backup and then each of the subsequent incremental backups.

Ultimately, except that Btrfs doesn’t “visually” indicate which is the “parent” snapshot, you have to go into each “Brtfs sub show” snapshot and look for the one that doesn’t have a parent (it’s the father) to restore it first, then the others.
On an incremental backup of the partition / it can be a lot, as it’s constantly moving, especially in rolling, and can become very uncertain or tedious. To restore a multi-boot layout quickly, it’s not the best solution.

Another thing is that btrbk, which makes incremental backups, doesn’t seem to ensure natively that a valid backup-parent exists each time it makes an incremental backup. (or prevent to remove him accidently)
In fact, it’s the user-defined retention rules that govern everything. So I imagine that, with a parameter error, you could very well inadvertently delete the famous parent backup.

For the reasons given above, please tell me if I’m wrong (restoration time for multiple incremental backups, uncertainty as to whether the backup-parent accompanies them properly), I’m interested in a reliable solution on this point, and I have the impression that the simplest solution is a full backup with btrbk, but given the size of my system, two questions arise

  • how often? how many copies should I keep?
  • how to configure btrbk so that it only backs up the root volume at least,
    and not incremental snapshots.

Thank you very much for your answers.

2

I am not that familiar with btrbk, I use snapper.

If you are using remote btrfs snapshots for your backup strategy, space shouldn’t be a huge issue unless you have a very long retention period or your data is very large to begin with.

In general, you are better off taking and sending snapshots more frequently than less frequently.

Eh…what? Unless btrbk is doing something very strange(which I doubt), you should not need to do this. A btrfs snapshot contains a reference to all the data. You should only ever need to restore one snapshot in the event of total data loss.

None of this makes any sense to me. I think you are misunderstanding how snapshots and snapshot replication works. The snapshots are sent incrementally but it isn’t an incremental backup like you are thinking about.

With snapshots, there is very little harm in taking them more frequently. I take snapshots hourly.

The thing that matters most is how long you retain your snapshots.

In other words, given a typical desktop usage pattern, if you retain a weeks worth of data, the space used won’t be much different if you have 100 snapshots in that week or if you only have one that is a week old. This should be true unless you have a very high rate of data turnover where you both add and delete large data files very frequently.

I am not I understand what you are trying to accomplish here.

3

I use btrbk for daily incremental backups on my second hard drive. Creating a first backup takes a long, then creating a second backup goes much faster because of the deduplication.

btrbk only supports Btrfs, no other filesystem. However, one problem is that if you get Btrfs update with new bugs in future, it would corrupt two Btrfs on two separate disks when using the same rolling release OS except two separate different OS. Is btrbk a backup strategy bad for you?

I would recommend using two different filesystems on two separate disks on the same computer.

I use restic for weekly incremental backups on ZFS mirror on my other multiple disks via REST API. Of course, two separate Linux kernels run in parallel on my same computer. The first kernel runs in my main system without ZFS module, the second kernel LTS with ZFS module runs on KVM/QEMU with access to ZFS pool on the physical disks as passthough storage.

4

@dalto , as you can see, the first btrbk snapshot is non-incremental here @home_sid

Backup Summary (btrbk command line client, version 0.32.6)

Date:   Sat Jun 10 22:03:11 2023
Config: /etc/btrbk/btrbk.conf

Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
— deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)

/mnt/btr_pool/@endeavour
+++ /mnt/btr_pool/btrbk_snapshots/@endeavour.20230610T2203

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@endeavour.20230610T2203

/mnt/btr_pool/@home_endeavour
+++ /mnt/btr_pool/btrbk_snapshots/@home_endeavour.20230610T2203

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@home_endeavour.20230610T2203

/mnt/btr_pool/@home_sid
+++ /mnt/btr_pool/btrbk_snapshots/@home_sid.20230610T2203
*** /run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@home_sid.20230610T2203

the other(endeavour) are later snapshots, and are incremental, unlike btrfs snapper-snapshots

5

btrfs snapshots are btrfs snapshots. It doesn’t matter which tool you use to take them. Even if they are sent incrementally, they still represent 100% of the data at that point in time.

In point of fact, there is not really much of a difference between a snapshot and subvolume. Snapshots are subvolumes on btrfs.

6

hi ,

this first snapshot has no parents

[root@falke-macbookair72 d3b7825f-42a2-44f8-994d-08511e0996b9]# btrfs su show @home_sid.20230610T2203
@home_sid.20230610T2203
Name: @home_sid.20230610T2203
UUID: fb32e242-0046-f142-9e0c-9b5ed3b4f9e7
Parent UUID: -
Received UUID: 938c443c-6a67-2b42-8f8a-c2efb6d06077
Creation time: 2023-06-10 22:03:15 +0200
Subvolume ID: 299
Generation: 522
Gen at creation: 514
Parent ID: 5
Top level ID: 5
Flags: readonly
Send transid: 9847
Send time: 2023-06-10 22:03:15 +0200
Receive transid: 519
Receive time: 2023-06-10 22:05:51 +0200

I will consider it as A snapshot , then modify something snap B , modify something snap C and then
deleting something from A , and trying if recover everything by restoring C

7

I understand how it works. But it doesn’t really have much meaning. A parent just tells you where it comes from. It is just metadata.

The parent is just telling you where the snapshot lives. If you move the snapshot, the parent ID will change. Although, the UUID should stay the same.

8

yet , I have done this , modified nothing in @home_sid

rerun btrbk run

[root@falke-macbookair72 d3b7825f-42a2-44f8-994d-08511e0996b9]# btrbk run

Backup Summary (btrbk command line client, version 0.32.6)

Date:   Sat Jun 10 22:18:38 2023
Config: /etc/btrbk/btrbk.conf

Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
— deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)

/mnt/btr_pool/@endeavour
+++ /mnt/btr_pool/btrbk_snapshots/@endeavour.20230610T2218

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@endeavour.20230610T2218

/mnt/btr_pool/@home_endeavour
+++ /mnt/btr_pool/btrbk_snapshots/@home_endeavour.20230610T2218

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@home_endeavour.20230610T2218

/mnt/btr_pool/@home_sid
=== /mnt/btr_pool/btrbk_snapshots/@home_sid.20230610T2203

it detected, there is nothing to do , there was no backup created … xxx2203 whas the last one ===

9

The point is, you don’t need to worry about parents and incremental vs full. That is just diagnostic text btrbk is sharing with you. It doesn’t change what is actually happening.

Every snapshot both remote and local contains 100% of the data, you don’t restore the first one and then the rest afterwards.

In fact, even if you delete the original one and everything but the latest, that latest one will still contain all the data.

2 Likes
10

create a file in @home_sid/falke/desktop
and reruning btrbk run :

Backup Summary (btrbk command line client, version 0.32.6)

Date:   Sat Jun 10 22:24:51 2023
Config: /etc/btrbk/btrbk.conf

Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
— deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)

/mnt/btr_pool/@endeavour
+++ /mnt/btr_pool/btrbk_snapshots/@endeavour.20230610T2224

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@endeavour.20230610T2224

/mnt/btr_pool/@home_endeavour
+++ /mnt/btr_pool/btrbk_snapshots/@home_endeavour.20230610T2224

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@home_endeavour.20230610T2224

/mnt/btr_pool/@home_sid
+++ /mnt/btr_pool/btrbk_snapshots/@home_sid.20230610T2224

/run/media/falke/d3b7825f-42a2-44f8-994d-08511e0996b9/@home_sid.20230610T2224

and

this one has its parent subvolume (the first one )

root@falke-macbookair72 d3b7825f-42a2-44f8-994d-08511e0996b9]# sudo btrfs sub show @home_sid.20230610T2224
@home_sid.20230610T2224
Name: @home_sid.20230610T2224
UUID: 9d872485-f77b-a94d-81ed-a11bf4c96d2c
Parent UUID: fb32e242-0046-f142-9e0c-9b5ed3b4f9e7
Received UUID: 7a438594-6373-d246-90dc-f78fa6a73052
Creation time: 2023-06-10 22:24:53 +0200
Subvolume ID: 304
Generation: 537
Gen at creation: 536
Parent ID: 5
Top level ID: 5
Flags: readonly
Send transid: 10954
Send time: 2023-06-10 22:24:53 +0200
Receive transid: 537
Receive time: 2023-06-10 22:24:53 +0200
Snapshot(s):

this time a snapshot and an external backup are created

11

I am not sure I follow your point from all that data.

12

ok , I will follow on tomorrow,

i will review my tests and make a summary of it

13

You never need to restore more than one snapshot. You can restore the one snapshot which most closely suits your needs and be done with it. This idea you have about parents is not right.

That’s because it doesn’t matter. You may delete any snapshot you wish at any time in any order, and the remaining snapshots will be fine.

Most commonly, people delete whatever snapshots are the oldest when they wish to free up some space or reduce the number of snapshots being saved for some reason. Then, they carry on making new snapshots and deleting the oldest, and on and on. This notion of preserving “parent” snapshots is not typical–I have to guess you are thinking of something else.

Don’t forget, btrbk will not back up your EFI partition where all your kernels and initramfs images live (because that partition is not Btrfs). If you need to roll back after a kernel upgrade breaks something, for example, this backup strategy will not be enough.

1 Like
14

Don’t forget, btrbk will not back up your EFI partition where all your kernels and initramfs images live (because that partition is not Btrfs). If you need to roll back after a kernel upgrade breaks something, for example, this backup strategy will not be enough.

yes, thank you for mentioning it, I’m well aware of that.

for the rest, btrfs does introduce a notion of incremental backups (I didn’t say incremental snapshots on the local disk, and I think btrbk just uses it).

see this page in French (je suis français , point 4 incremental snapshots.

According to my tests, when I do two incremental snapshots, if I haven’t changed anything on the sub-volume, no snapshot is taken.
If I create a file, I get a snapshot that includes not just the element created, but all the files. I agree with you that it’s not an incremental backup, but an image. What is incremental, on the other hand, is being able to restore an image to a given day, and being able to retrace the situation between two snapshots.

The only “incremental” thing I can see in btrbk’s behavior at the moment is that it won’t take a snapshot if it doesn’t detect any change on a sub-volume, thus preserving a little space. I’ll now see if I can restore a snapshot C having deleted the so-called parent (A)

15

good,

I’ve just restored backup (brtbk) C (the last one) without first restoring backup A and it works perfectly.

I wonder why btrbk claims to make incremental backups .
The only thing it does is to NOT taking snapshots if nothing has changed on the sub-volumes.
I wonder what its behavior might have to do with the behavior of differential backups in the link I mentioned above.

I’ll test, just for my curiosity the incremental backup, mentioned in the link, to see how it reacts. With command line.

PS : for my efi partition backups, I use a script that makes a copy of /efi on the root partition before making a snapper snapshot (I also use snapper).
I don’t use the kook possibilities of pacman , since my to other distros are Debian-based.

16

btrbk should normally take an atomic snapshot of its subvolume on your partition where your btrfs system is currently running. After this snapshot is taken, it will be sent to another partition or disk by btrfs send | receive like what btrbk does. This is called an incremental backup to the other partition or disk.

17
  • why? this name is not appropriate
Incremental Operation
During incremental backup, we make a new snapshot:

   btrfs subvolume snapshot -r /home /home/BACKUP-new
   sync
We can now send the difference between the old and new backup to the backup volume:

   btrfs send -p /home/BACKUP /home/BACKUP-new | btrfs receive /backup/home
Once this command completes, we should have these 4 subvolumes: /home/BACKUP, /home/BACKUP-new, /backup/home/BACKUP and /backup/home/BACKUP-new. We will now need to migrate the new backup as the old one, and do something for the old one. We could keep it around, maybe timestamped with the date of that backup, or just straight out delete it. Here, I am deleting it:

   btrfs subvolume delete /home/BACKUP
   mv /home/BACKUP-new /home/BACKUP
   btrfs subvolume delete /backup/home/BACKUP
   mv /backup/home/BACKUP-new /backup/home/BACKUP
But for instance, if you did want to keep a history of backups, perhaps you would snapshot one of the snapshot directories with something like:

   btrfs subvolume snapshot -r /backup/home/BACKUP /backup/home.$(date +%Y-%m-%d)
This concludes the incremental backup step.

this i an appopriate incremental backup. Seemingly you didn’t see my link above or here :

https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Incremental_Backup.html

18

You posted here:

This is similar to what I wrote, plus a backup should be on a separate hard drive.

The outdated wiki just explained in general terms. There is the current official document: https://btrfs.readthedocs.io/en/latest/Introduction.html

19

It doesn’t make incremental backups. It sends the snapshots incrementally.

It isn’t actually creating backups at all in the traditional sense. It makes a local snapshot and then replicates it out to the remote target. When the data is sent to the remote target, only the incremental data is sent.

It is basically a fast way to create and maintain a replica.

Eh, this one is interesting. Back in the days of tape backups, incremental backup had a very specific meaning. However, since deduplicating backup solutions took over, incremental now is usually defined as any backup that only sends new data since the last backup.

1 Like
20

@dalto

It isn’t actually creating backups at all in the traditional sense. It makes a local snapshot and then replicates it out to the remote target. When the data is sent to the remote target, only the incremental data is sent.

  • I totally understand that

but, i thougt incremental is the difference , meaning:

basis + delta = is complete actual volume, I must miss something.

@zesko i’ve seen its outdated but not mandatory deprecated,

thanks for the good linkk, I will have a look.