New iSO BTRFS newbie here

I feel I really need to study this more closely, please bear with me:sweat_smile:

This is from my fstab:

UUID=4e33451a-4483-4d0a-9fee-c48e63991416 / btrfs rw,noatime,compress=lzo,ssd,autodefrag,discard=async,space_cache,subvolid=256

So if I change the suvolid= to point to a snapshot and boot into it, can the “resulting” system be considered as rolled back and I can use it as normally, install/remove software, modify things etc.?

As long as the snapshot/subvolume is rw then, yes.

Of course, if you then boot back into subvolid=256 none of those changes will be there.

As a side note, I think you consider using subvol= instead of subvolid=. It helps from a recovery perspective. This is because as you can see it @anon49550872’s example above it is very easy to change the name of a subvolume. So if you want to swap out subvolumes to boot from it is easier to change their names than it is modify your bootloader to boot a different subvolid.

This is what I had in mind when i asked the initial question if an snapshot taken by snapper, which is ro, could be “converted” to rw. And then use its id or name to boot into.

Sure. I thought this as a way of restoring the system in case the system with root at subvolid=256 is broken beyond repair.

Alright. I have them usually specified by their name. Not sure why I changed it to id. I will change it back.

I guess there is not much of a gain after all doing this than doing it as described above by @anon49550872.

In my opinion the most common reason to boot into a snapshot is so you can restore a snapshot without having to boot off an ISO.

1 Like

@anon49550872
I can’t read the end of the line on the 12th line? is it sudo btrfs subvolume snapshot /mnt/@old/.snapshots/3/snapshot/ /mnt/@

It should be. Just be careful to replace 3 with the actual snapshot you want to restore.

What about the ones that aren’t numbers? Are they all numbered?

Yes, they all have a number. For example, here is a portion of the output you posted earlier in the topic.

  # | Type   | Pre # | Date                        | User      | Cleanup  | Description                                                              | Userdata
-----+--------+-------+-----------------------------+-----------+----------+--------------------------------------------------------------------------+---------
  0  | single |       |                             | root      |          | current                                                                  |         
  1  | single |       | Tue 17 Aug 2021 11:11:49 PM | ricklinux | timeline | AfterInstall                                                             |         
 80  | single |       | Fri 20 Aug 2021 12:00:01 AM | root      | timeline | timeline                                                                 |         
111  | single |       | Sat 21 Aug 2021 12:00:22 AM | root      | timeline | timeline                                                                 |         
125  | single |       | Sun 22 Aug 2021 12:00:28 AM | root      | timeline | timeline                                                                 |         
151  | single |       | Mon 23 Aug 2021 12:00:06 PM | root      | timeline | timeline                                                                 |         
164  | single |       | Tue 24 Aug 2021 12:00:35 AM | root      | timeline | timeline                                                                 |         
175  | pre    |       | Tue 24 Aug 2021 01:58:05 PM | root      | number   | pacman -Syu                                                              |         
176  | post   |   175 | Tue 24 Aug 2021 01:58:06 PM | root      | number   | endeavouros-theming firefox openssl run-parts                            |         
179  | single |       | Tue 24 Aug 2021 04:00:23 PM | root      | timeline | timeline                                                                 |         
180  | single |       | Tue 24 Aug 2021 04:41:32 PM | root      | number   | boot                                                                     |         
181  | pre    |       | Tue 24 Aug 2021 04:44:40 PM | root      | number   | pacman -S --config /etc/pacman.conf -- extra/docbook-xml extra/itstool e |         
182  | post   |   181 | Tue 24 Aug 2021 04:44:40 PM | root      | number   | asciidoc docbook-xml docbook-xsl glib2-docs gobject-introspection gtk-do |         
183  | pre    |       | Tue 24 Aug 2021 04:44:59 PM | root      | number   | pacman -U --noconfirm --config /etc/pacman.conf -- /home/ricklinux/.cach |         
184  | post   |   183 | Tue 24 Aug 2021 04:45:00 PM | root      | number   | libpamac                                                                 |         
185  | single |       | Tue 24 Aug 2021 05:00:21 PM | root      | timeline | timeline                                                                 |         

That number in the first column is the snapshot number.

Yes, I understand. What I wanted to say by gain was in terms of the work and time needed to get it done and skipping the live ISO. I wanted to cut some corners by doing it through /etc/fstab, but then

So, would that mean it could be possible to change the root= in /etc/default/grub and regenerate the grub.cfg?

I don’t use grub much anymore. Does grub have a root=? I thought it autodiscovered the root.

Either way, what you need to change is the rootflags=subvol=

1 Like

You are right! It normally has UUID=root= (or is it root=UUID=?).
Anyways

This is it! Thank you!

1 Like

I was having dinner, otherwise I would have answered you.

I have stored the commands used in text file, follows what I have stored:

Restore a snapshot:

1 List which snapshot you will be using
snapper -c root list (listing root but could be home or other snapper configuration)
2 Boot to live USB
3 Mount BTRFS partition:
sudo mount /dev/sda4 /mnt (should be the BTRFS volume used for root, /)
4 sudo mv /mnt/@ /mnt/@old
5 sudo btrfs subvolume snapshot /mnt/@old/.snaphots/1/snapshot /mnt/@ (number 1 should be the snapshot choosen in step 1)
6 sudo mv /mnt/@old/.snapshots /mnt/@/.
7 sudo btrfs subvolume delete /mnt/@old

I’ll leave again, will drink some saque (japanese drink made with rice), wanna get drunk today, its friday :stuck_out_tongue:

1 Like

Sake please :pray:

3 Likes

@anon49550872 so you successfully used snapper for snapshots ? Maybe you can write some short guides for other newbies :slight_smile:

It worked, let me share my .txt file for you guys, it has everything I know so far…
This .txt file is far from complete, there is still a lot to learn…

Especial thanks to @ricklinux, @dalto and @pebcak who helped me a lot


The instrctuions below are for BTRFS filesystem, snapper usage for snapshots management, and snapper-rollback or rollback through Live USB.
Intended for single disk only which means no RAID instructions.

If you are reading this before installing the system, I suggest you to follow the procedures to have a separate @snapshots subvolume and have the suggested filesystem layout as Calamares itself won't do it for you.
This is required for snapper-rollback to work.
https://wiki.archlinux.org/title/Snapper#Suggested_filesystem_layout
This is only valid for endeavouros-2021.08.27-x86_64.iso or previous, as we don't know yet how the next Calamares fstab and mount.conf will be.

*** Installation and configuration

** LIVEUSB STEP #1
edit /etc/calamares/modules/mount.conf
Add these to the end of the file: (Make sure it is written exactly as the already existings mountpoints).
- mountPoint: /.snapshots
subvolume: /@snapshots

Now, it also has been recommended by the EnOS community, to change the compression from LZO to ZSTD:

** LIVEUSB STEP #2
edit /etc/calamares/modules/fstab.conf
Change the mountOptions for btrfs compression field from lzo to zstd as per below:
mountOptions:
ssdExtraMountOptions:
	btrfs: compress=zstd    

Then install the system, once it's finished, proceed below:

# pacman -S snapper

Configuration of snapper and mount point
It is assumed that the subvolume @ is mounted at root /. It is also assumed that /.snapshots is not mounted and does not exist as folder, this can be ensured by the commands: 

# umount /.snapshots
# rm -r /.snapshots

Then create a new configuration for /. Snapper create-config automatically creates a subvolume .snapshots with the root subvolume @ as its parent, that is not needed for the suggested filesystem layout, and can be deleted. 

# snapper -c root create-config /
# btrfs subvolume delete /.snapshots
Delete subvolume (no-commit): '//.snapshots'

After deleting the subvolume, recreate the directory /.snapshots.

# mkdir /.snapshots

Now mount @snapshots to /.snapshots. Where # is the BTRFS root partition.

# mount -o subvol=@snapshots /dev/sda# /.snapshots
To make this mount permanent, add an entry to your fstab.
Or if you have an existing fstab entry remount the snapshot subvolume:

# mount -a

# chmod 750 /.snapshots/
Give the folder 750 permissions.
This will make all snapshots that snapper creates be stored outside of the @ subvolume, so that @ can easily be replaced anytime without losing the snapper snapshots. 

**  SNAPPER configuration steps for @ (/).

# sudo snapper -c root create-config / (if you edited mount.conf in calamares and performed all the steps above, this is not needed, jump to next step).
# systemctl enable --now snapper-timeline.timer
# systemctl enable --now snapper-cleanup.timer

** Set snapshot limits

Now that snapper is installed and partially configured, you can change the snapshot limits and snapshot and cleanup frequencies.
As per Arch Wiki, the default settings will keep 10 hourly, 10 daily, 10 monthly and 10 yearly snapshots. You may want to change this in the configuration, especially on busy subvolumes like /

Here is an example section of a configuration named config with only 5 hourly snapshots, 7 daily ones, no monthly and no yearly ones:

/etc/snapper/configs/root

TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="5"
TIMELINE_LIMIT_DAILY="7"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="0"
TIMELINE_LIMIT_YEARLY="0"

*** SNAPPER useful commands:

- List snapper configs
snapper list-configs

- List snapshots for a config where config_name is the name of the configuration:
snapper -c config_name list

- Delete a snapshot:
snapper -c root delete N  (config is the name of the config and N is the number of the snapshot)
snapper -c root delete 65 70 (delete snapshot 65 and 70)
snapper -c root delete 65-70 (delete snapshot from 65 to 70)
Note: When deleting a pre snapshot, you should always delete its corresponding post snapshot and vice versa.

- Remove a config from snapper:
The sequence of the commands here is very important.

$ snapper -c config_name list
$ snapper -c config_name delete 3-18 (3-18 is the range from 3 to 18)
$ snapper -c config_name delete-config

- Remove root config from snapper cannot be done as described in the previous section, follows below how to accomplish it:

$ cd /etc/snapper/configs/
# rm root
edit /etc/conf.d/snapper and remove root from the quotes of SNAPPER_CONFIGS="root"

- Create a snapshot:

To take a snapshot of a subvolume manually, do:
# snapper -c config create --description desc
The above command does not use any cleanup algorithm, so the snapshot is stored permanently or until deleted.

To set a cleanup algorithm, use the -c flag after create and choose either number, timeline, pre, or post. number sets snapper to periodically remove snapshots that have exceeded a set number in the configuration file. For example, to create a snaphot that uses the number algorithm for cleanup do:
# snapper -c config create -c number

- Restore a snapshot Using snapper-rollback: (Suggested SNAPPER filesystem layout required) Live USB boot not required.

Find the device used for the root partition through:
$ mount | grep btrfs 
Configure snapper-rollback
Edit /etc/snapper-rollback.conf and add to the end of the file:
dev = /dev/sda# 
Where # is the number of the drive for the root partition

Then, to rollback, find out which snapshot you want to rollback to (snapper -c root list) and then issue the following command: 
# snapper-rollback #
Where # is the number of the snapshot you want to rollback to.

- Restore a snapshot: (Default calaremes filesystem layout) Live USB boot required.

List which snapshot you will be using
# snapper -c root list
Boot to live USB
Mount BTRFS partition:
# mount /dev/sda# /mnt (should be the BTRFS volume used for root, /)
# mv /mnt/@ /mnt/@old
# btrfs subvolume snapshot /mnt/@old/.snaphots/#/snapshot /mnt/@
Where # is the number of the snapper snapshot you wish to restore. Your / has now been restored to the previous snapshot. 
# mv /mnt/@old/.snapshots /mnt/@/.
# btrfs subvolume delete /mnt/@old  - Optional, to remove the current broken snapshot
Now just simply reboot. 

- Restore a snapshot: (Suggested SNAPPER filesystem layout) Live USB boot required.

List which snapshot you will be using
# snapper -c root list
Boot to live USB
Mount BTRFS partition:
# mount /dev/sda# /mnt (# should be the BTRFS volume used for root, /)
# mv /mnt/@ /mnt/@old
# btrfs subvolume snapshot /mnt/@snapshots/#/snapshot /mnt/@
Where # is the number of the snapper snapshot you wish to restore. Your / has now been restored to the previous snapshot. 
# btrfs subvolume delete /mnt/@old - Optional, to remove the current broken snapshot
Now just simply reboot. 

*** BTRFS useful commands:

- List subvolumes:
# btrfs subvolume list /

- List subvolumes, print the parent ID
# btrfs subvolume list -p /

- Count snapshots:
# btrfs subvolume list -ts / | wc -l

- Display disk usage:
# btrfs filesystem usage /
$ btrfs filesystem df /

- List of all the btrfs filesystems on the systems and which devices they include
# btrfs filesystem show

- List details about a subvolume (using root in the example below)
# btrfs subvolume show /

- Create subvolume
# btrfs subvolume create /path/dir

- Delete subvolume
# btrfs subvolume delete /path/dir

- Change a snapshot from ro (read only), to read and write: where # is the number of the snapshot, useful for deleting files from snapshots and other actions.

Verify status:
$ btrfs property get /.snapshots/#/snapshot
ro=true

Change to rw:
# btrfs property set /.snapshots/#/snapshot ro false

Fallback to ro:
# btrfs property set /.snapshots/#/snapshot ro true

You can now modify files in /path/to/.snapshots/<snapshot_num>/snapshot like normal. You can use a shell loop to work on your snapshots in bulk. 


*** Tips:

** Preventing slowdowns

- updatedb
By default, updatedb (see mlocate) will also index the .snapshots directory created by snapper, which can cause serious slowdown and excessive memory usage if you have many snapshots. You can prevent updatedb from indexing over it by editing:
/etc/updatedb.conf
PRUNENAMES = ".snapshots"

** Access for non-root users

Each config is created with the root user, and by default, only root can see and access it.
To be able to list the snapshots for a given config for a specific user, simply change the value of ALLOW_USERS in your /etc/snapper/configs/config file. You should now be able to run snapper -c config list as a normal user.
Eventually, you want to be able to browse the .snapshots directory with a user, but the owner of this directory must stay root. Therefore, you should change the group owner by a group containing the user you are interested in, such as users for example:

# chmod a+rx .snapshots
# chown :users .snapshots

*** BTRFS Maintanance:

- SCRUB / 
I personally prefer to run scrub manually, note that a scrub in / works at filesystem level, so all subvolumes included, I suggest to run it once per month.

# btrfs scrub start /
# btrfs scrub status /

- TRIM (SSD), don't forget to enable TRIM in fstab (discard=async), or enable periodic fstrim.timer as per below:

- Enable timer
# systemctl enable --now fstrim.timer
# systemctl status fstrim.timer

- Start fstrim manually
# systemctl start fstrim.service
# systemctl status fstrim.service


- If you are using the provided systemd timers, you can edit them to change the snapshot and cleanup frequency. 

- snapper-timeline.timer
# systemctl edit snapper-timeline.timer
to revert changes:
# systemctl revert snapper-timeline.timer

When editing snapper-cleanup.timer, you need to change OnUnitActiveSec. (I prefer 12h period, default is 24h).
to revert changes:
# systemctl revert snapper-timeline.timer


*** Troubleshooting

- Orphaned snapshots causing wasted disk space
It is possible for snapshots to get 'lost', where they still exist on disk but are not tracked by snapper. This can result in a large amount of wasted, unaccounted-for disk space. To check for this, compare the output of

# snapper -c <config> list
to
# btrfs subvolume list -o <parent subvolume>/.snapshots  (sudo btrfs subvolume list -o /.snapshots/ to check root)

Any subvolume in the second list which is not present in the first is an orphan and can be deleted manually. 

- Balance:

Balance on a single-device filesystem a balance may be also useful for (temporarily) reducing the amount of allocated but unused (meta)data chunks. 
This will run on filesystem level, thus all subvolumes included.

# btrfs balance start /

- Checksum hardware acceleration
CRC32 is a new instruction in Intel SSE4.2. To verify if Btrfs checksum is hardware accelerated:

# dmesg | grep crc32c
[    3.630588] Btrfs loaded, crc32c=crc32c-intel, zoned=yes
If you see crc32c=crc32c-generic, it is probably because your root partition is Btrfs, and you will have to compile crc32c-intel into the kernel to make it work. Putting crc32c-intel into mkinitcpio.conf does not work. 

- Check the mount options to confirm if everything is correct:

$ mount | grep btrfs

- Check if the SSD optimizations were set:
# dmesg | grep ssd

Useful links:
https://forum.endeavouros.com/t/first-try-with-btrfs-questions/17353/2
https://btrfs.wiki.kernel.org/index.php/Gotchas#Fragmentation
https://wiki.archlinux.org/title/Snapper#Creating_a_new_configuration
https://wiki.archlinux.org/title/Btrfs
https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_optimized_for_SSD.3F
https://man.archlinux.org/man/btrfs.5#MOUNT_OPTIONS
https://wiki.archlinux.org/title/Btrfs#Creating_a_subvolume
https://bbs.archlinux.org/viewtopic.php?id=194491
https://wiki.archlinux.org/title/Systemd#Editing_provided_units
https://wiki.archlinux.org/title/Systemd/Timers
https://github.com/jrabinow/snapper-rollback

6 Likes

Thanks for sharing you findings so far! Thanks to you and your thread I have learnt a great deal myself, something I am grateful for. Bookmarked your post.

2 Likes

This is awesome :+1::sunglasses: thank you a lot

2 Likes

Uau…

61 snapshots, almost no disk usage…

[marcelo@eos ~]$ sudo btrfs subvolume list -ts / | wc -l
61
[marcelo@eos ~]$ sudo btrfs filesystem usage /
Overall:
    Device size:		 125.41GiB
    Device allocated:		  15.02GiB
    Device unallocated:		 110.39GiB
    Device missing:		     0.00B
    Used:			  13.94GiB
    Free (estimated):		 110.91GiB	(min: 110.91GiB)
    Free (statfs, df):		 110.91GiB
    Data ratio:			      1.00
    Metadata ratio:		      1.00
    Global reserve:		  38.05MiB	(used: 0.00B)
    Multiple profiles:		        no

Data,single: Size:14.01GiB, Used:13.49GiB (96.32%)
   /dev/sda4	  14.01GiB

Metadata,single: Size:1.01GiB, Used:458.84MiB (44.46%)
   /dev/sda4	   1.01GiB

System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
   /dev/sda4	   4.00MiB

Unallocated:
   /dev/sda4	 110.39GiB

Wasn’t there supposed to be some sort of performance issue associated with having many snapshots?

I was getting periodic system hangs when using BTRFS. The primary culprit was the btrfs-cleaner, but I never did much digging to nail it down.

Thanks for doing the legwork.

I also recently got my laptop as btrfs thanks to the new EOS ISO and you saved me a lot of work!

The Arch wiki which I did first wasn’t the best since it starts with making a config with “config” as the name and later they mention the same setup line but with “root” in the name.

Was able to undo it luckily since the wiki does mention the paths to the files so I was able to remove those and start fresh.

2 Likes