Moving to Endeavour & dealing with current partition setup

General system EndeavourOS installation
1

So basically, I am contemplating going all in this weekend and installing over Manjaro on this PC, but I have a few questions related to how to best deal with my current partition setup and how to move data over seamlessly.

This is my current setup.

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 232.9G  0 disk 
├─sda1   8:1    0   511M  0 part /boot/efi
├─sda2   8:2    0    50G  0 part /
└─sda3   8:3    0 182.4G  0 part /home
sdb      8:16   0   1.8T  0 disk 
└─sdb1   8:17   0   1.8T  0 part /mnt/hdd
sdc      8:32   0 223.6G  0 disk 
└─sdc1   8:33   0 223.6G  0 part /run/timeshift/backup


inxi -p
Partition: ID-1: / size: 48.97 GiB used: 32.06 GiB (65.5%) fs: ext4 dev: /dev/sdb2 
           ID-2: /boot/efi size: 510.0 MiB used: 280 KiB (0.1%) fs: vfat dev: /dev/sdb1 
           ID-3: /home size: 178.52 GiB used: 70.48 GiB (39.5%) fs: ext4 dev: /dev/sdb3 
           ID-4: /mnt/hdd size: 1.79 TiB used: 562.65 GiB (30.7%) fs: ext4 dev: /dev/sda1 
           ID-5: /run/timeshift/backup size: 219.06 GiB used: 159.98 GiB (73.0%) fs: ext4 dev: /dev/sdc1

Basically, I have a 250 GB SDD for /boot/efi, /, and /home, but I have my Documents, Pictures, Videos, Music, etc. directories on and symlinked to a 2TB HDD. I also have some backups on that HDD, as well as my Steam library.

I have removable SSD I took from an old PC to act as my Timeshift backup drive.

I am wondering if it would be best to start fresh and get rid of this divided /home and symlinked folders setup, maybe plop /home on my HDD altogether, or if I should preserve the setup I have now.

Secondly, I really want to give btrfs a try, especially for snapshots, in case my first weeks with Endeavour prove to have some teething problems. I am however not interested in encrypting this machine, as it is a desktop.

Does it make sense to still have /, /boot/efi/, /home and all that on separate partitions and have all these as btrfs? I assume if you do this, you could selectively only roll back each partition independently of each other?

As a fail safe, I am actually consider nipping out and buying another SSD and HDD, so that even if I totally mess it up somehow, I can just pop my Manjaro drives back in…but that might be expensive.

Anyhow, hope you get the gist! Any advice welcome.

2

The symlinked folders model is a great model, I would keep that. It will make your transition here much easier. Just blow away /dev/sda and preserve the rest.

/boot/efi should be seperate and stay as vfat/fat32.

/ and /home should be in a single btrfs partition. btrfs has a concept of subvolumes that allow you to further break down your data.

I am guessing you are going to find this is actually easier you think.

3 Likes
3

That’s a relief to hear! The only issue I have found with this setup is that some backup solutions don’t play nice with symlinked directories, so I end up having to make a separate backup of all those symlinked directories on the HDD and a separate one for whatever is still on /home on my SSD, rather than just backing up /home in one fell swoop.

Oh yeah, duh. Just thinking…could I just use the exact partition structure I have now, e.g. 510.0 MiB for /boot/efi and just overwrite it in the EndeavourOS installer?

Ah, ok - so in my case, I would wipe / and /home, making one 227.48 GiB combined partition on btrfs? Or could I just leave /home as ext4 and only make / btrfs? I have heard people recommend not putting /home on btrfs, especially if you run VMs and such (I depend on them for work).

I suuuuree hope so… :sweat_smile:

4

Just posted this in another thread:
Please note: If you’re planing on using Timeshift for making snapshots later on make sure to set up your subvolumes correctly.

Timeshift’s BTRFS snapshots are supported only on BTRFS systems having an Ubuntu-type subvolume layout (with @ and @home subvolumes).

For a really quick way to set up btrfs and timeshift maybe check the following post which describes how to adapt the BTRFSonLUKS wiki article to not use encryption.

5

You could.

You could do that too if you prefer.

This is one of those situations where there is no “right” answer. :grin:

1 Like
6

Oh…ok - btrfs shapshots are actually done through Timeshift? I was under the impression that btrfs snapshots and Timeshift snapshots were a seperate thing.

So would this preclude me from ONLY setting up / as btrfs, and leaving /home as ext4? What would the advantage be of setting up /home as a btrfs subvolume?

Thanks for the writeup by the way, I have to admit it looks pretty daunting at first glance, but maybe I will be able to crack it with some more time.

7

Timeshift actually just uses standard btrfs tools for snapshot creation. You could also ceate these yourself without Timeshift from the cli. But Timeshift really simplifies this process immensely.

No, you could always still mount your ext4 partition to /home by changing the fstab later on.

Well you could also take snapshots of your home data and easily restore. I use this feature when I don’t want any “test residue” staying in /home after playing around with stuff.
I also run quite a few VM’s and haven’t had any issues regarding btrfs. Anyway, you could always store and run these from a mounted ext4 partition if need be.

[Edit] Another advantage: the subvolumes /root and /home share the same space. So your /root will never have too much (wasted) or too little space.

1 Like
8

Thanks.
It only looks that way; it’s meant to basically just be copied and pasted to the terminal. Should be (meant to) easy to set up.

2 Likes
9

Shows you how smooth Timeshift works that I didn’t even realise it was using btrfs (though now on thinking about it, I do remember something about this when I first configured it).

Ok, so here’s what I did - since I’m keeping my HDD as a place to symlink stuff like ~/Documents, etc. I just moved my VMs there for now, as well as any dotfiles from my home that might come in handy. That way, presumably, I can just do what you recommend and make one btrfs partition for / and /home. My HDD is on ext4, and it can stay there, and I can still play with btrfs on sda.

But in the Linux world that’s like taking candy from strangers! :laughing:

10

@2000 or @dalto - would you kindly advise me if I still need to carry out step #04 – Mount newly installed btrfs system to temporary directory /mnt/btrfs, even if I am not encrypting this install? I notice the instructions include:

sudo blkid -o device | grep luks

Edit:

From the guide below

11

Yes, you do need to mount, but the “grep luks” part won’t work if you’re going the unencrypted route.

I’d advise you follow the “quick” version (BTRFSonLUKS copy&paste) and change according to this post.
The outcome will be identical to the wiki article you’re referencing.

Adapting that Wiki article would of course also work; just haven’t done that yet.

[Edit] You can always add other partitions and mount via fstab after the install.

1 Like
12

Ah ok, switched over to the copy and paste guide.

For this step:

sudo grub-mkconfig -o /boot/grub/grub.cfg

I got:

grub-probe: error: cannot find a GRUB drive for /dev/sdd1.  Check your device.map.
done

sdd1 however is the USB stick I am installing from, so I suppose that’s probably expected.

This is the output I got from:

sudo mkinitcpio -p linux

=> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 5.8.1-arch1-1
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
loadkeys: Unable to open file: ie: No such file or directory
  -> Running build hook: [filesystems]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build: 5.8.1-arch1-1
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: aic94xx
==> WARNING: Possibly missing firmware for module: wd719x
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
loadkeys: Unable to open file: ie: No such file or directory
  -> Running build hook: [filesystems]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-fallback.img
==> Image generation successful

Again, probably ok, but we’ll find out when I reboot and start step #6.

13

Well - looks like I am now on EndeavourOS! Everything seems to be working correctly, though I’m not sure how to check if that btrfs stuff is all working.

4 Likes
14

See below.
You’ll notice the benefits of btrfs snapshots once you create and restore them. Unlike rsync, restoring btrfs snapshots takes about 2 seconds!

Some useful btrfs related commands:

  • show mounted subvolumes
    findmnt -nt btrfs

  • show mountpoints (including fat32 and ext4 partitions)
    findmnt -D -t btrfs,vfat,ext4

  • list all subvolumes
    sudo btrfs sub list /

  • check size
    sudo btrfs filesystem usage /

3 Likes
15

Great, will check these tomorrow in more detail. Thanks so much for talking me through the setup, as well as for putting together that guide. It’s a very valuable resource it seems.

2 Likes
16

Coming back to my experience of installing in the light of the next day, since I switched to the copy and paste version halfway through the install yesterday, I just read through the verbose version again to get my head wrapped around what I did. I was feverishly copying and pasting stuff, probably a lot more casually than I should have been (seems I have less and less time lately after work), so I was worried I might have missed running a command or something that could cause problems further down the line.

I think the verbose version is great, though I can see that it’s not quite as clear with the action points, i.e. the commands aren’t always ready to run. It is still very useful having an explanation for what you’re doing with each command. It makes the whole think feel more ‘linuxy’ and less like you have to hide what you’re doing from the Arch purists watching over the fence in disgust :eyes:. A fusion of the two approaches would be perfect, or even just carrying over small explanatory paragraphs to the copy and paste version. Not being ungrateful though, I really appreciate all the time that went into it.

Running through the commands you provided me in the last post:

I get these results, all of which, at least to my inexperienced eye, seem alright:

/                       /dev/sda2[/@]                     btrfs  rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=256,subvol=/
├─/run/timeshift/backup /dev/sda2                         btrfs  rw,relatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=5,subvol=/
├─/swap                 /dev/sda2[/@swap]                 btrfs  rw,relatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=269,subvol=
├─/home                 /dev/sda2[/@home]                 btrfs  rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=257,subvol=/
└─/var/cache/pacman/pkg /dev/sda2[/@var-cache-pacman-pkg] btrfs  rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=268,subvol=/

It looks like all the various sub-volumes of / are accounted for and mounted in the right places:

findmnt -D -t btrfs,vfat,ext4
SOURCE                            FSTYPE   SIZE  USED  AVAIL USE% TARGET
/dev/sda2[/@]                     btrfs  232.4G 26.7G 204.6G  11% /
/dev/sda2[/@home]                 btrfs  232.4G 26.7G 204.6G  11% /home
/dev/sda2[/@swap]                 btrfs  232.4G 26.7G 204.6G  11% /swap
/dev/sda2[/@var-cache-pacman-pkg] btrfs  232.4G 26.7G 204.6G  11% /var/cache/pacman/pkg
/dev/sda1                         vfat     510M  696K 509.3M   0% /boot/efi
/dev/sdb1                         ext4     1.8T  624G   1.1T  34% /mnt/hdd
/dev/sda2                         btrfs  232.4G 26.7G 204.6G  11% /run/timeshift/backup

Here’s all the subvolumes listed (interesting to see that /var/lib/portables and /var/lib/machines are automatically created - seems to be something to do with systemd):

sudo btrfs sub list /
ID 256 gen 3183 top level 5 path @
ID 257 gen 3183 top level 5 path @home
ID 260 gen 22 top level 256 path var/lib/portables
ID 261 gen 23 top level 256 path var/lib/machines
ID 268 gen 3053 top level 5 path @var-cache-pacman-pkg
ID 269 gen 108 top level 5 path @swap

I couldn’t find anything concrete on why the ID skips from 261 to 268, but I’ll do more digging.

Here’s the size info:

 sudo btrfs filesystem usage /
 Overall:
     Device size:                 232.39GiB
     Device allocated:             28.02GiB
     Device unallocated:          204.37GiB
     Device missing:                  0.00B
     Used:                         26.64GiB
     Free (estimated):            204.59GiB      (min: 204.59GiB)
     Data ratio:                       1.00
     Metadata ratio:                   1.00
     Global reserve:               52.88MiB      (used: 0.00B)
     Multiple profiles:                  no
 
 Data,single: Size:26.01GiB, Used:25.79GiB (99.15%)
    /dev/sda2      26.01GiB
 
 Metadata,single: Size:2.01GiB, Used:877.45MiB (42.68%)
    /dev/sda2       2.01GiB
 
 System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
    /dev/sda2       4.00MiB
 
 Unallocated:
    /dev/sda2     204.37GiB

Lots of unallocated space, but from my reading into btrfs, seems routine.

Checking to see if my swapfile is there, it seems that’s all ok too:

              total        used        free      shared  buff/cache   available
Mem:          16003        1946       12476         222        1580       13539
Swap:          8191           0        8191

And also shows up here:

cat /proc/swaps
Filename                                Type            Size            Used            Priority
/swap/swapfile                          file            8388604         0               -2

Admittedly, even after going through this, there is the urge to reinstall and only run each command when I fully understand it, but I also don’t want to jinx what appears to be a perfectly functional system. Plus, if this was all incorporated in the regular install automatically, I would never noticed it doing all this in the background. I think it might just mean I’m ready for a more Arch-style install process next time around.

But hey, at the end of the day: I’m on EndeavourOS now, looking forward to learning more about my system, this distribution and the community. Thanks again for helping me get there @2000.

3 Likes
17

I wish i understood it too! My setup is less elaborate as it is the copy and paste only on a fresh install so i don’t have some of the mnt points you have.

[ricklinux@eos-xfce ~]$ findmnt -D -t btrfs,vfat,ext4
SOURCE                                                        FSTYPE   SIZE  USED  AVAIL USE% TARGET
/dev/mapper/luks-efc1098d-46e9-4d34-b4bf-a1b77b3a2361[/@]     btrfs  465.3G 84.5G 380.2G  18% /
/dev/nvme0n1p1                                                vfat     511M    2M   509M   0% /boot/efi
/dev/mapper/luks-efc1098d-46e9-4d34-b4bf-a1b77b3a2361[/@home] btrfs  465.3G 84.5G 380.2G  18% /home
/dev/mapper/luks-efc1098d-46e9-4d34-b4bf-a1b77b3a2361[/@swap] btrfs  465.3G 84.5G 380.2G  18% /swap
/dev/mapper/luks-efc1098d-46e9-4d34-b4bf-a1b77b3a2361[/@var-cache-pacman-pkg]
                                                              btrfs  465.3G 84.5G 380.2G  18% /var/cache/pacman/pkg
/dev/mapper/luks-efc1098d-46e9-4d34-b4bf-a1b77b3a2361         btrfs  465.3G 84.5G 380.2G  18% /run/timeshift/backup
[ricklinux@eos-xfce ~]$
1 Like
18

I think mine is different, mainly because I have a separate 2TB hard drive mounted on /mnt/hdd - that’s where I store all my image files, documents, etc. So at least theoretically, reinstalling and such should be easier.

One thing I noticed is I don’t seem to have the option in grub to boot into snapshots, though the “limit-timeshift-snapshots-update-grub.service” is loaded and active, and I also seem to have 4 snapshots (one created as 1st and three others before upgrading).

19

I also have a 4 TB hard drive but i don’t know how to add that as a mnt? It has to be done before using the copy and paste method? You had an existing setup so i’m not sure and under normal circumstances i think you just add it to the fstab? I’m not that good at some of this stuff without putting a lot of effort into it which i struggle with time and (effort)! :point_left:

20

Nope, I actually did it afterwards, I added it manually into my fstab.

I found the UUID of the drive (for me it was sdb1).

sudo blkid

Then I used nano to edit my fstab, adding in this line:

UUID="51596fec-9d35-4522-87c5-64bacce1314c" /mnt/hdd     ext4   errors=remount-ro 0 1

Obviously my UUID, my desired mountpoint, the filesystem (ext4 in this case) would be different than yours probably. I also can’t say for sure if the flags at the end should be the same for you.

I’m not an expert by any means, but that’s just how I did it.

EDIT:

On further inspection, I think the last field should be 2, rather than 1. But again, please don’t take that as gospel.