Calamares not using boot subvolume - user_commands.bash

Hi folks,

I have been tinkering with user_commands.bash for a few weeks now. It’s great! It works really well and makes it easy to put together a very customized installation.

I have become stumped on a small matter, and I think it is possible I have drifted outside the bounds of what Calamares supports but I thought I would check and see if anyone can spot something I have missed.

I have been renaming the default Btrfs subvolumes in the pre-installation section of user_commands.bash like this:

	# Use the new subvolume name, set mount options
	sed -i -e "s|@home|${subvolume_name}_home|" \
	       -e "s|@cache|${subvolume_name}_cache|" \
	       -e "s|@log|${subvolume_name}_log|" \
	       -e "s|@|${subvolume_name}|" \
	       -e "s|defaults, compress=zstd:1|noatime, compress=zstd|g" /etc/calamares/modules/mount.conf

It works great, it reliably creates the subvolumes with the names I want and mounts them correctly, and adds the correct subvol= mount option for each on into /etc/fstab.

Recently, I started testing a setup which calls for a separate boot partition. I want /boot in a subvolume for snapshotting purposes, and also so multiple installations can share the same boot partition. I added this, to cut in another pair of mountPoint and subvolume lines into /etc/calamares/modules/mount.conf:

    # Add new lines for /boot subvolume to mount.conf
    sed -i "/btrfsSubvolumes:/a \    - mountPoint: /boot\n      subvolume: /${subvolume_name}_boot" /etc/calamares/modules/mount.conf

This is also working great. Here is a snippet from /etc/calamares/modules/mount.conf after the installation has completed:

[...]

btrfsSwapSubvol: /eos-swayswap

btrfsSubvolumes:
    - mountPoint: /boot
      subvolume: /eos-sway_boot
    - mountPoint: /
      subvolume: /eos-sway
    - mountPoint: /home
      subvolume: /eos-sway_home
    - mountPoint: /var/cache
      subvolume: /eos-sway_cache
    - mountPoint: /var/log
      subvolume: /eos-sway_log

mountOptions:
    - filesystem: default
      options: [ noatime ]
    - filesystem: efi
      options: [ fmask=0137, dmask=0027 ]
    - filesystem: btrfs
      options: [ noatime, compress=zstd ]
    - filesystem: btrfs_swap
      options: [ noatime ]

In this example, subvolume_name is set to eos-sway.

I thought adding these lines would be enough for this to work, because in this post the user appears to have successfully added quite a few custom subvolumes.

In my case, it doesn’t work. The subvolume does not get created. I thought perhaps Calamares cannot create this subvolume because it is not one of the default ones, which is fair enough, so I added a few additional lines in the pre-installation part of the script to create the subvolume manually (so Calamares only has to mount it).

    # Define the boot partition
    boot_partition="boot_partition_placeholder"

    # Mount the boot partition
    echo "Mounting the boot partition..."
    mount -o subvolid=0 "${boot_partition}" /mnt
    
    # Create the boot subvolume
    echo "Creating the boot subvolume..."
    btrfs subvolume create "/mnt/${subvolume_name}_boot"

    # Unmount the boot partition
    echo "Unmounting the boot partition..."
    umount /mnt

This is working, in that the code is run and the subvolume gets created and everything…but Calamares is still not using it. /boot gets mounted on the partition, not the subvolume. /etc/fstab is not getting the subvol=eos-sway_boot mount option (everything else about the /boot line in fstab is correct). All the kernels and initramfs images get dumped outside the subvolume like this:

ls -l /mnt
total 180140
drwxr-xr-x 1 root root      328 Apr 16 10:11 arch_boot
drwxr-xr-x 1 root root        0 Apr 16 16:28 eos-sway_boot
-rw------- 1 root root 58448315 Apr 16 16:41 initramfs-linux-fallback.img
-rw------- 1 root root 17584060 Apr 16 16:41 initramfs-linux.img
-rw------- 1 root root 57039092 Apr 16 16:41 initramfs-linux-lts-fallback.img
-rw------- 1 root root 17397360 Apr 16 16:41 initramfs-linux-lts.img
-rw-r--r-- 1 root root  8099840 Mar 12 13:13 intel-ucode.img
-rw-r--r-- 1 root root 12927488 Apr 10 21:47 vmlinuz-linux
-rw-r--r-- 1 root root 12952064 Apr 13 07:50 vmlinuz-linux-lts

(arch_boot is a boot subvolume from another installation.)

The main thing that is different with this setup as compared to the other user’s thread I linked is /boot is a separate partition, like this:

sda 
├─sda1    <-- EFI
├─sda2    <-- boot (Btrfs)
└─sda3
  └─live-btrfs_crypt <-- root (Btrfs)

That is to say, I have two Btrfs partitions. It seems possible Calamares supports managing subvolumes on a single filesystem only, I’m not sure.

If I move the kernels and images from the root of /boot into the subvolume I made and add the missing subvol= option in to /etc/fstab after the installation completes, the installation boots fine and everything works just as intended. Those are fairly simple little tasks I could incorporate into a script that runs post-installation or something like that. Still, it seems a little hacky; I figured before I do that I should at least ask and see if anyone knows how to get it to work, or if it is even possible.

Here is the installation log: https://termbin.com/m6mx

The full user_commands.bash script is here: https://gitlab.com/BluishHumility/eos-sway/-/blob/testing/user_commands.bash

2 Likes

I tested a couple installations where I left the second partition (the boot partition) out of it, and just used the EFI partition and the encrypted Btrfs partition. I kind of just wanted to see if I would stumble on any clues; this setup would not actually work in practice because /boot would be on the encrypted partition (and I’m not using Grub or setting up a keyfile, or any of that).

First, I kept the same custom subvolume layout above (including the boot subvolume) but only used the one Btrfs partition mounted at /. This installation failed almost immediately; it appears to run into an issue mounting the partitions (install log: https://termbin.com/nuwe).

However, when I checked the Btrfs partition afterward it did create all the subvolumes (including the boot subvolume). So: the custom subvolume layout in /etc/calamares/modules/mount.conf is being honored.

Next I installed on just the EFI partition and encrypted Btrfs partition again, but I left off the boot subvolume altogether (so /boot is just included in the root subvolume). In this case the installer got pretty far, but eventually failed because Calamares tries to set up a key file but can’t do it (I assume because I didn’t set up the encryption through Calamares).

    Starting job "Configuring LUKS key file." ( 17 / 38 ) 
2024-04-17 - 21:29:09 [6]: virtual Calamares::JobResult LuksBootKeyFileJob::exec()
    There are 1 LUKS partitions 
2024-04-17 - 21:29:09 [6]:     .. root "/dev/nvme0n1p3" passphrase? false 
2024-04-17 - 21:29:09 [6]:     .. No root passphrase. 
2024-04-17 - 21:29:09 [6]: virtual void Calamares::JobThread::run()
    Skipping non-emergency job "Write LUKS configuration for Dracut to /etc/dracut.conf.d/calamares-luks.conf" 

[...]

    ERROR: Installation failed: "Encrypted rootfs setup error" 
2024-04-17 - 21:29:10 [6]:     .. - message: "Encrypted rootfs setup error" 
2024-04-17 - 21:29:10 [6]:     .. - details: Root partition /dev/nvme0n1p3 is LUKS but no passphrase has been set.

Which is fair enough–like I said, that setup wouldn’t have worked anyhow without setting up a keyfile and properly configuring the bootloader. It seems Calamares is just trying to be helpful. Installation log: https://termbin.com/3s48

I’m not sure those two tests were incredibly informative, but it is rather curious that the installation crashed almost immediately when I asked for the boot subvolume, but got much further when I did not. :thinking:

Finally, just to test setting up a custom boot subvolume through the installer in general I did an installation with no encryption, and no separate Btrfs partition for /boot, only adding the the boot subvolume to /etc/calamares/modules/mount.conf. In this test, everything works as expected–the boot subvolume was added, kernels and images were installed inside the subvolume, correct mount options in /etc/fstab, etc.

The handling of encrypted devices where Calamares has not set up the encryption is somewhat unpredictable, but to be fair I am using a hacky workaround to set that up in the first place (this one: https://github.com/calamares/calamares/issues/1564#issuecomment-948979591). Officially I think it is not supported.

The custom subvolume layout in mount.conf does work just fine, with a boot subvolume or probably any other subvolume, if all the subvolumes are being installed on the root partition. Something about a custom subvolume being on a separate filesystem seems to not be handled correctly for some reason (again, perhaps it is not supported).

At this point I think I’ll just write some kind of post-install cleanup script for dealing with the way the boot partition is handled in the original scenario and call it Good EnoughTM.

Calamares only supports creating subvolumes on the partition mounted at /. It can’t create subvolumes in other partitions.

Gotcha. That’s what I thought, but good to hear it from the horse’s mouth so to speak. Thank you @dalto!

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.