Am I right in assuming there's no painless way to migrate one's installation to a new disk?

Got me a new M.2 NVME SSD so wanting to switch over.

I’ve been searching for a bit, saw some entries on Arch wiki and other places.

If there isn’t a simple solution, I’d probably rather just go with a fresh install on the disk.

(Also, please save the spiels about how “painless/simple isn’t always the best” :grin: I know some of you can’t seem to help yourselves.)

formatting disks create new UUID partitions that required /etc/fstab updated and Grub re installed

The command line tool dd could be used to clone an entire disk:

https://wiki.archlinux.org/title/Dd#Cloning_an_entire_hard_disk

It should be relatively painless unless one uses the wrong if and of hence overwriting the wrong disk. That’s painful!

And as always, do your own research and use it at your own risk :wink:

1 Like

Questions:

  1. Is your current drive formatted as btrfs?
  2. Can you add the new SSD in addition to your current drive?
  1. No, ext4. I don’t mind switching, but haven’t looked into how that would affect me yet.
  2. Yes

I see it mentions MBR but how does it work with UEFI? Same?

Not painless.

However.
If you back up your /home you can reuse a lot of config files. I especially suggest backing up the entire .mozilla folder if you use Firefox and everything under .config that is for an app you will install again.

1 Like

I don’t think it should make any difference there.

I had an idea, but it requires btrfs to have any chance of working.

As far as I know, the least painful way of migrating is a fresh install.

Before migrating, create a list of install packages using:

yay -Qqe > pkglist.txt

After migrating, restore your packages using:

yay -S - < pkglist.txt

Note, I’m using yay here since a) it comes installed on Endeavour and b) it will capture and reinstall packages from the AUR.

If your not worried about AUR packages pacman would work just as well.

2 Likes

It is possible (probably).
It is possible to convert whole disk (e.g. /dev/sda) to an image and then copy this image to a new disk. Then you have to adjust partition size (that is probably the most risky part).

I did some modification based on this link to create a full image backup for my raspberry pi - partitions, UUIDs are kept, only thing that has to be fixed is partiton size. When you do dd to a slightly different disk you may run into a trouble.
This script is designed for raspberry pi but you can take it as a starting point.

But on the other hand you can just backup your /home and /etc (if you modified it) and just do a fresh install.

My modification if you are interested
#!/bin/bash
#
# Copyright 2016 - Matteo Mattei <matteo.mattei@gmail.com>
# https://www.matteomattei.com/how-to-shrink-raspberry-pi-backup-images/
# This script is intended to be used to shrink raspberry pi images
# created with dd client command.

# Check if you are root
if [ ! $(id -u) -eq 0 ]; then
    echo "ERROR: This program must run as root"
    exit 1
fi

# Check if all necessary dependencies are installed
if [ -z "$(which dd 2> /dev/null)" ]; then
    echo "ERROR: dd command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which parted 2> /dev/null)" ]; then
    echo "ERROR: parted command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which losetup 2> /dev/null)" ]; then
    echo "ERROR: losetup command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which resize2fs 2> /dev/null)" ]; then
    echo "ERROR: resize2fs command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which truncate 2> /dev/null)" ]; then
    echo "ERROR: truncate command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which gzip 2> /dev/null)" ]; then
    echo "ERROR: gzip command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which 7za 2> /dev/null)" ]; then
    echo "ERROR: 7za command not found - please install it and retry"
    exit 1
fi
if [ -z "$(which zerofree 2> /dev/null)" ]; then
    echo "ERROR: zerofree command not found - please install it and retry"
    exit 1
fi


# argument parse - TODO
SOURCE="${1}"
IMG="${2}"


dd if=${SOURCE} of=${IMG} bs=64M status=progress


# Check if the file exists
if [ ! -f ${IMG} ]; then
    echo "ERROR: File ${IMG} does not exist"
    exit 1
fi


# Synchronize cached writes to persistent storage
sync

# add filesystem extend script to /boot/cmdline.txt to run it during first boot
BOOT_START=$(parted ${IMG} unit B print | grep -i fat | awk '{print $2}')
BOOT_SIZE=$(parted ${IMG} unit B print | grep -i fat | awk '{print $4}')

BOOT_START=${BOOT_START::-1}
BOOT_SIZE=${BOOT_SIZE::-1}

BCKP_TMP="bckp_tmp"

mkdir ${BCKP_TMP}
mount -v -o offset=${BOOT_START},sizelimit=${BOOT_SIZE} -t vfat ${IMG} ${BCKP_TMP}

sed -i '$s|$| init=/usr/lib/raspi-config/init_resize.sh|' "${BCKP_TMP}/cmdline.txt"

umount ${BCKP_TMP}
rm -r ${BCKP_TMP}


# Create loopback file to modify ext4 partition
INFO=$(parted -m ${IMG} unit B print | grep ext4)

NUM=$(echo ${INFO} | awk -F':' '{print $1}')
START=$(echo ${INFO} | awk -F':' '{print $2}')
OLD=$(echo ${INFO} | awk -F':' '{print $3}')
DUMMY=$(echo ${INFO} | awk -F':' '{print $4}')

START=${START::-1}
OLD=${OLD::-1}

LOOPBACK=$(losetup -f --show -o $START $IMG)
e2fsck -p -f ${LOOPBACK}
if [ ! ${?} -eq 0 ]; then
    echo "ERROR: filesystem seems corrupted"
    losetup -d ${LOOPBACK}
    exit 1
fi

# Zero garbage blocks from old deleted files to reduce size of final 
zerofree -v ${LOOPBACK}

# Shrink ext4 partition
INFO=$(resize2fs -P ${LOOPBACK} 2>&1)
SIZE=$(echo ${INFO} | awk -F': ' '{print $2}')
SIZE=$((${SIZE} + 1024))

resize2fs -p ${LOOPBACK} ${SIZE}

losetup -d ${LOOPBACK}
SIZE=$(( ${SIZE} * 4096 + ${START} ))

parted ${IMG} rm ${NUM}
parted -s ${IMG} unit B mkpart primary ${START} ${SIZE}

SIZE=$(( ${SIZE} + 58720257 ))
truncate -s ${SIZE} ${IMG}

# Make a compressed image
echo "compressing image..."
7z a -t7z -m0=lzma2 -mx=9 ${IMG}.7z ${IMG}
rm ${IMG}
echo "done"
1 Like

I guess if you had LVM installed you could add the new disk as a new physical volume to the group, and then move the logical volumes from the old disk(PV) to the new one. But i think /boot is outside LVM and probably would involve a new grub install

I did this more than a year ago - it isn’t painless.
I moved from spinning rust to M2 NVME.
I had/have multiple distros installed - so the objective was to migrate all installations.
In theory just replicating the partitions using dd should work - assuming that the new ssd is the same size or larger than the old disk.
<edit>But I used partclone to copy partitions
Stating the obvious - but better safe than sorry - when cloning, none of the partitions should be mounted/ in use.
I complicated things by expanding some of the partitions.
I keep an installation of supergrubdisk2 handy, and that made it less painful to boot the new partitions when I messed up.
Once booted in the EndeavourOS - I reinstalled grub
You may also have to do something like,

grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=CHANGEME

HTH
Good luck.

1 Like

First time is not painless, practice in a VM first can help, subsequent times it takes only minutes.

Create partitions for new location.

From live environment …

– Use partclone to backup your root file system.

– Use partclone to restore your root file system to new root partition.

– Use uuidgen and tune2fs to change the restored file system UUID.

– Chroot into new root file system and change any old UUID references in /etc/default/grub, /etc/fstab, /etc/crypttab, etc.

– For EFI systems mount efi partition in chroot and re-install grub.

– Regenerate initramfs’ using mkinitcpio -P

– Use efibootmgr to ensure new boot order is correct.

Reboot into your system in a new location.

I’ve cloned my system many times using this technique. Seems complicated but it really is not.

I always keep post update (usually weekly) partclone images of all my systems on backup media. I can use these for disaster recovery, ie disk failure, or to restore if the main system becomes borked. With recovery omit the UUID editing stuff.

If you have a fully encrypted system simply open the luks container with cryptsetup before running partclone. On your new location luks encrypt your root partition first. Include the new luks container UUID in the UUID editing steps above.

Also, rsync .

3 Likes

Thanks all, I think I have the answer to my Q. I’m gonna file this under the “things that will likely be simpler in a decade” category.

As I said in the main post, I’ll prob just go with a new EOS installation, along with just making a list of apps I have + backing up some config data.

1 Like

If you are interested, you can try the following:

  1. Generating package list for programs installed from the OFFICIAL repositories
    In the following example I’m using -Qqe option, but you can manually edit this list or make your own with only desired packages
    pacman -Qqe >installed_packages.txt
    or
    pacman -Qe | awk ‘{print $1}’ > installed_packages.txt
    (Edit the generated list as you wish using your prefered text editor)

and

  1. Generating package list for programs installed from the AUR packages
    NOTE: this actually lists any package external from official repositories, so EndeavourOS packages are also listed here.
    pacman -Qqm >aur_packages.txt

then

Installing to another computer or reinstalling to your current one
–>First copy the generated .txt file(s) to the Desktop. Open Terminal and type < cd Desktop >. Then run the one or more of the following:
There are multiples ways to accomplish this installation; here are a couple of examples:
–>For official Arch repositories, if you have one package by line in installed_packages.txt, you can:

  1. sudo pacman -S $(cat installed_packages.txt |xargs) --needed --noconfirm
    Or even:
  2. for x in $(cat installed_packages.txt) do sudo pacman -S $x --needed –noconfirm

and

–>For the AUR, if you have one package by line in aur_packages.txt, you can:
yay -S $(cat aur_packages.txt |xargs) --needed --noconfirm


–needed will only install new packages, so you won’t waste time reinstalling the already installed
–noconfirm won’t prompt the confirmation step “Do you want to install?”. Recommended for both methods but
especially for the second method and with huge list of files

But, better yet, (and this is what I do)

  1. OR merge both generated files into one new text document [Create Document] (named
    complete_installed_packages.txt) and install with yay instead of pacman.
    yay -S $(cat complete_installed_packages.txt |xargs) --needed --noconfirm

I must thank fernandomaroto who was the person who sent these instructions to me. I have “cloned” several computers onto which I have installed EnOS and then, using these instructions, obtained “duplicates” of my original computer.

Believe it or not, it “reads” more complicated than it is and it is really easy to accomplish (in my opinion).

Lawrence

2 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.