Which Backup / Imaging System for 2022?

Lounge EndeavourOS pub
#1

HMS_EndeavourFrance

1m

I have a poll on file systems and it got a bit off topic, so I promised to create a topic of its own, because backing up is far too important a subject these days to just casually mention it as something you should do, but for some reason not too many seem to have reliable backups/images when disaster finally strikes – and it will, trust me.

mrvictoryDev ISO tester

HMS_Endeavour

7h

Which backup system in 2022: Timeshift, Snapper, Borg, CloneZilla, dd, btrfs send/receive, ZFS (zpool?) etc​:joy: Choose 1!

@dalto opened up a can of worms for me, although I am sure it is just his analytical mind racing way past me. So I want to know which system one and why, and which filesystem is it best suited for which?


dalto

8h

There are many but my recommendation is always something backed by borg. It is easy enough to setup without a GUI using something like borgmatic. If you prefer a GUI, you can use a borg frontend like vorta.

On the other hand, replicating snapshots is actually a handy tool.

I would take either of those options over a disk image though others would disagree.

I would like to know those arguments against disk images and why. Having a boot ready image is how I used my various Macs for the past few decades. I never lost a backup, but had plenty of crashes, particularly in those days of unreliable hardware.

Personally, I backup the home folder with PikaBackup using Borg backend to an external drive. I am so paranoid I also do a Déja Dup to an external drive, and upload it to to 2 cloud locations. But having BTRFS Assistant with Snapper on a BTRFS setup, gives me much more confidence, since it has saved me many times over just when tinkering and messing something up to make it unbootable. Just a fast selection on the grub menu and we are back in business in moments.

So, what do think are the good and bad arguments for imaging packages like CloneZilla, even good old dd is powerful and simple which will save you in a pinch.

Let us all know what you think…

2 Likes
#2

Clonezilla 1/month
Snapper snapshotting hourly (/, /home, /games)
btrfs send/receive daily (home-grown app)
mksquashfs weekly (my VMs)

While having the Clonezilla image gives me peace of mind, it’s the hourly snapshots that I rely on most.

3 Likes
#3

Rescuezilla → Once per month. This is very handy tool, basically Clonezilla with GUI
Rsnapshot → to autobackup backup /etc and /home once/week
Chezmoi → to sync some of my dotconfigs between machines

1 Like
#4

I use timeshift for system snapshots (daily). Then, if something breaks, I have
a bootable image to run timeshift and restore the files.
Probably not the most efficient/elegant way, but it is simple to set up (even for someone with little knowledge on filesystems, like me :joy:) and it gets the job done.
But, I have yet to consider a tool to backup my home folder. Currently, I’m just storing some .dotfiles in github (using this method).

#5

Snapshots are not backup.

I don’t use system snapshots – I consider them to be a waste of space, because I can fix pretty much any issue myself (ultimately, reinstalling the OS is always an option). I used to use Timeshift with rsync for system snapshots, but some two years ago I stopped with that, realising it was unnecessary and wasteful, and that the only part of the filesystem that must be backed up is the home directory (and Timeshift is terrible for that).

I use borgmatic to back up my home directory to another local (internal) HDD. I also use about ten external HDDs for long term backup, where I just copy the backup database (and do a checksum), or even simpler, just copy the files. I do that for my music and video collection, which is larger than my local HDDs and for files I wish to keep, but I don’t use that often. I also have a rather bad habit of hoarding data, dozens of my external drives contain junk files I just can’t be bothered to delete, even though I’m never going to use them.

In addition to that, my most important files (i.e. files for which the thought of losing them is unbearable) are burnt on DVDs, in multiple copies, and these copies are given to trustworthy friends and family for storage.

I’ve learnt the importance of backup the hard way. I’ve had multiple HDDs and SSDs fail – it’s a fact of reality that all hardware eventually fails, and when that happens to HDDs or SSDs, there is typically data loss. Also, there always exists a chance of a user error leading to massive data loss, as well as software doing something malicious or stupid (one accidental space character in a rm command in some script can wipe out the entire home directory, for example).

When something like that happens, one is very happy to have a backup.

1 Like
#6

I do agree 100 % with this statement. I consider snapshots of the system partition unnecessary.

For anything else I have zfs and use frequent send/receive to an external raidz2 system.

I also do backups of other PCs over the network and if they do not have zfs I use rsync.

2 Likes
#7

The problem with using just rsync backing up corrupted files without realising they are corrupted. I’ve used rsync for backup for years and it never happened to me, but I can easily imagine a scenario where one important file gets damaged (maybe by a user error, doesn’t have to be any serious filesystem corruption) and rsync overwrites the good version of that file on the backup drive, without you even realising it. Then this file is pretty much gone, and when you eventually realise it there is a chance you won’t be able restore it from any previous backup (if corruption happened a long time ago).

#8

This is not an rsync problem, but a problem with the filesystem. zfs and btrfs can identify corrupted files because they use checksums on data and metadata. But no other filesystem can do that.

So for other filesystems, like ext4 or xfs, rsync is the best tool for backups. Many other GUI backup programs rely on rsync in the background. And if you a paranoid you can ask rsync to use checksums for the file transfer so that you can be 100 % sure that source and destination file are exactly the same.

2 Likes
#9

It has saved me more than once, but as @dalto and @kresimir point out snapshots are not complete backups.

#10

So if I am getting this right. BTRFS Snapper is not a complete backup, but BTRFS is still good because it can do checksums on data and metadata.

So which disk imaging tool would be good as a secondary strategy?

#11

True, that was already in my real experience. My RAM without “ECC” was already damaged after buying, I’ve been using full EXT4 for 9 months without noticing that corruption data has actually appeared, but without crashing system.
Then I bought a new SSD for Btrfs installation first time and used it, I noticed very easily the I/O error when copying or rsync BackUp to my USB stick. Btrfs journal log showed the backup was corrupted.
I thought Btrfs had the bug, but that’s not true, it was the faulty RAM after my hard research. MemTest64 can not help to detect the error in this RAM, that is why I send this RAM to PassMark company that tried to improve MemTest64, but it is very difficult to detect the error of this RAM.

rsync is not problem, but the corrupted backup was created by zip compression due to the bad RAM. That is why Btrfs automatically stopped running rsync and lets you get noticed very easily. Ext4 can not

#12

I am not using a disc imaging tool like Clonezilla or such. I am using zfs send/receive functionality to transfer a complete zfs dataset (incl. all snapshots) to a backup location. btrfs also has send/receive functionality and can do the same.

#13

I remembered you are have zfs, and will study up btrfs send/receive.

So in my case, I am now thinking to resize the / btrfs partition and create a ext4 as a safety precaution, since I now understand snapshots are not full system backups and I only have BTRFS partition like so:

Here is what I have now:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 465.8G  0 disk 
├─sda1   8:1    0   300M  0 part /boot/efi
├─sda2   8:2    0   8.8G  0 part [SWAP]
└─sda3   8:3    0 456.7G  0 part /home
                                 /var/cache
                                 /var/log
                                 /

sudo btrfs filesystem show /dev/sda3
Label: none  uuid: f73391ec-4b7d-4e1a-bc43-bc2be61b4507
	Total devices 1 FS bytes used 107.45GiB
	devid    1 size 456.68GiB used 114.06GiB path /dev/sda3

# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=E10A-465C                            /boot/efi      vfat    umask=0077 pri=10 0 2
UUID=981a41b7-e800-45fb-b749-ba8ae32234a3 swap           swap    defaults,noatime 0 0
UUID=f73391ec-4b7d-4e1a-bc43-bc2be61b4507 /              btrfs   subvol=/@,defaults,noatime,noautodefrag,compress=zstd 0 0
UUID=f73391ec-4b7d-4e1a-bc43-bc2be61b4507 /home          btrfs   subvol=/@home,defaults,noatime,noautodefrag,compress=zstd 0 0
UUID=f73391ec-4b7d-4e1a-bc43-bc2be61b4507 /var/cache     btrfs   subvol=/@cache,defaults,noatime,noautodefrag,compress=zstd 0 0
UUID=f73391ec-4b7d-4e1a-bc43-bc2be61b4507 /var/log       btrfs   subvol=/@log,defaults,noatime,noautodefrag,compress=zstd 0 0

Is this possible/easy/advisable?
How to add a ext4 to the fstab?

Thanks @mbod you always make sense :smiley:

1 Like
#14

I must say, the initial post is pretty messy and confusing to me with all the links and such… :stuck_out_tongue_winking_eye:

Now we are talking file system, plus snapshots plus backups. This is going to confuse the hell out for newbies. :scream:

I use ext4 and nextcloud to backup my data to my server with raid plus external drive. Never had corrupted files and no problem with ext4. At this moment in time I don’t see the need to use Btrfs or ZFS on any of my drives. But that might change, Btrfs seems complicated with all the volumes and such and I see too many posts with Btrfs problems in this forum, in most cases because users plunge into this head first without learning how it works and a system should be partitioned… ext4 is basically simple and idiot proof. Perfect for me.

I see the use case for Btrfs and ZFS, and followed news in past couple of years. I might give it a shot one day.

I don’t do system snapshots. I rather have a clean install or fix a problem myself. But I rarely have Linux crashing on me, lucky :grin: me

#15

I just backup my personal data and config files to external drives.

If my system breaks and I am able to repair it by myself or the help from community then that’s it.

If it breaks beyond repair, I would reinstall and restore my data and configs from the external drive.

Perhaps not the most sophisticated way of doing this but it works for me.

#16

I think you need to start with what your goals are. Otherwise you can’t come to a reasonable conclusion.

When it comes to recovering data, there are many different options which all have their own pros and cons. In the modern desktop world some examples would be:

  • Filesystem snapshots
  • One-way or bidirectional data sync
  • Versioned file backups
  • Disk images
  • Deduplicating disk-based backups

For server or enterprise backups there are even more options.

There also lots of different failure scenarios you need to consider(in no particular order):

  1. Loss of a physical drive
  2. Loss of an entire system
  3. Loss of your entire location
  4. File corruption
  5. Filesystem corruption
  6. An inadvertent or deliberate change to a file
  7. Accidental deletion of a file or multiple files
  8. A widespread system change that causes breakage

Snapshots can help with some of these scenarios. Namely, 4, 6, 7 and 8. If you replicate them, they can also help with 1, 2, and 5. They also have another advantage though. They are extremely inexpensive and can be taken with a high degree of frequency. This is why I ensure I have snapshots of almost everything. Because there is little reason not to.

I use my snapshots far more often than my actual backups. In fact, I hope to never have to use my backups. Unfortunately, my hopes and realities are not the same. :sweat_smile:

My personal strategy is that I use a combination of things depending on the nature of the data, the criticality and the situation in which I might want to recover the data. I use a combination of bidirectional sync, snapshots, snapshot replication and borg. I also rclone my borg backups to the cloud(I am considering changing this as there is some risk here). Of course, everything is encrypted. For critical data I have 3-4 copies of the data at 2 different locations. For less critical data I have at least 2 copies of the data.

Lastly, I would warn of the potential pitfalls of sync. I have seen people burned by sync too many times. Sync copies the current state of the file. If the file is corrupt, it will gladly overwrite a good copy with bad data. If your filesystem is corrupted, depending on the nature of the corruption, you may sync widespread corruption into your replica. If you make changes to a file that need to be reverted, sync can’t support that since it overwrites. Sync, when used alone, is rarely a good backup solution.

5 Likes
#17
  1. End of the :clown_face: :earth_africa: due to nuclear war

honka_animated-128px-14

2 Likes
#18

I am not sure I have a viable backup strategy to recover from that. :rofl: :clown_face:

1 Like
#19

We need a backup space capsule! :rocket:
:enos:

honka_memes-128px-26

3 Likes
#20

Maybe your favorite NTFS supports it :

:rofl: :rofl: :rofl:

1 Like