Before I update/update, I backup

_ i suffer from mental health issues and am typing this after recovering from a panic attack. i’m ok, just really trying not to freaking out & stressing. please be kind and helpful._

So this latest Grub issue really has be shaken up. Before I proceed I will state I am NOT opposed to totally reinstalling. I’m typing this from live USB image of EndeavourOS_Artemis_neo_22_7.iso

I remember reading about the benefits of btrfs over ext4. As a long time linux user I have always chosen ext4 but it would have been a pain to “convert” my previous install (nov.2021). Should I choose btrfs this time around?


Before I do anything I have reference this post How do I backup packages? - #2 by Kresimir

But how do I do this pacman -Qeq > pkglist.txt if i am using the ISO?


I came up with this rsync command for copying my /home to external SSD, I think I have all the right flags?
rsync --dry-run -avzhe --progress --exclude 'node_modules' [source] [destination]


Anything else I am missing?

1 Like

if the grub is broken, you need to use the liveusb or other form of grub to come intoo the game.

but that means you have to use chroot as change root in long. dont know how to get te list without comming in thats dificult. backup is good if you have a reaching point to enable te backup if you cant you have chroot in. things are broken if its unfixable, but chroot in and grub-install can do a lot.

The --progress, --exclude, source, and destination elements are all correct. (You might also want to exclude “.cache” - up to you?).

With “-avzhe”, you are telling it to:

  • -a keep all file permissions, symbolic links, file ownership, etc
  • -v verbose output - displays the details of the transfer
  • -z compress data during transfers to save space
  • -h show the numbers in the output in a human-readable format
  • -e instructs rsync to use the SSH protocol for remote transfers

If you are copying to an external SSD, then “-e” should not be necessary. You might also want to consider “-t” which preserves modification times.

Finally, “–dry-run” will show the output as a test, but you should of course remove this when carrying out the actual backup.

Hello, I share your same points of view about being backup-paranoid, and for what it’s worth I’m happy to share a few hints about my daily routines.

Not a manual, at all - nor even pieces of advice - just a few hints & links.

BACKUP: my 3 pillars are timeshift + rsync, and optionally btrfs.
I’ve customized the very same scripts to run on several machines, EFI or legacy bios, Intel or Amd, ext4 or btrfs.
I use dedicated physical disks for my rsyncs, plus a second unit for timeshifts in ext4 mode.

— About rsync backup —
I’m currently using the script ‘rsync_tmbackup.sh’, that I’ve customized for my needs and crontab-bed 4 times a day:
https://github.com/laurent22/rsync-time-backup

But I’ve happily used some ‘(em)BORG’ forks:
About BORG: https://borgbackup.readthedocs.io/en/1.1-maint/installation.html
i.e. NO $ sudo pacman -S borg, but
pip install borgbackup llfuse

see https://borgbackup.readthedocs.io/en/stable/installation.html#dependencies
and
https://emborg.readthedocs.io/en/latest/installing.html
and
https://emborg.readthedocs.io/en/latest/examples.html
and
https://emborg.readthedocs.io/en/latest/commands.html

— About btrfs & timeshift —
Convert to btrfs:
FEDORA guide: https://fedoramagazine.org/convert-your-filesystem-to-btrfs
TitusTech guide: https://christitus.com/btrfs-guide/
( Youtube BTRFS Guide | The Best Desktop File System - YouTube )

Also on Youtube, Stephen’s Tech Talks:
“Speedy Install EndeavourOS 2021.04.17 ArchLinux with BTRFS, TimeShift, Gnome 40, ZRAM, and PipeWire”

But above all, see “Timeshift and grub-btrfs in Linux Arch” by Lorenzo Bettini ( it’s in English )

More:
How To: GRUB-BTRFS Snapshot Booting on Arch Linux Systems

and

Regards, Piero

1 Like

I was able to generate the pkglist.txt by going in as arch-chroot. So I’m happy about that. It’s now safely in my backup drive.

After reading more about the benefits of btrfs I want to wipe/install the whole disk, size is 1TB :slight_smile: I love the idea of snapshots. QUESTION: if I had had btrfs, I wouldnt be in this mess now, would i? :thinking:

I previously had 500GB partition for my old Pop OS install, but I’m ok with wiping that clean as well.

I did run rsync as mentioned above. Did it twice in fact (two different destinations).

I’m thinking this time around I will keep a seperate partition for /home. I remember years ago, that was a benefit. QUESTION: Is that still true today?


I’m awaiting more comments, will perform a clean install this evening - hoping I have not forgotten anything.

    1. rsync home folder to external SSD (twice)
    1. created pkglist.txt (surprisingly small list at 309 lines)
1 Like

You are not in a mess. It is easily fixable and there are several threads on the forum on how to fix it. Snapshots on BTRFS may have made it easier depending on how you set up the snapshots. Long answer maybe short answer nope.

Thank you Piero for your help and scripts. They will be useful later when I have reinstalled and have btrfs set up :+1:

P.S.: running # timeshift --restore from a live usb terminal after arch-chrooting has perfectly fixed all my ext4 installations.

Guess/wish it can do as well on btrfs machines, good luck folks.
Piero

depend what is broke also -timeshift orwhatever you used works. but it brings whole system back in time :slight_smile: if you know what it is you do fix it on that thats different to keepmore up.

things are broken if it became unfixable… :slight_smile:
one update does not mean system breaks, it mayby unworkable but keeping an iso is a luxury just if you know mounting and chrooting , it can become faster then place backup back :slight_smile:

I’m getting ready to install.

Launched Gparted and this is what I have set up - seeking advice from more experienced folks before I carry on :+1:


Previously disk space was nearly 50/50 Pop OS and EndeavourOS. But this time around I want to give it the full amount, minus swap and boot partition.

UPDATE: installation went fine, am booted up perfectly and copying over the configs for bspwm, sxhkd, and polybar.

But first. am reading up on the links provided by /u/storm up above.

Thanks again all for the assistance during this difficult time.

Now I’m off to learn about brtfs and timeshift before I install anything :wink:

1 Like

hhmm, I know I selected btrfs while setting up the installer, but it didn’t work!


Should I go through the installer again?

I don’t see a reason why something like your operating system not booting would be enough to cause a panic attack.

If you think about it, you’ll realise that it is really not a big problem, it’s just the operating system. In the worst case scenario, you can reinstall it and have it back up and running in 20 minutes or less. But this is almost never the case – usually there is a post with the solution available before you even notice such a problem.

So, my advice is: don’t worry about a broken OS. You don’t really need snapshots, timeshift, and whatnot: it’s just an OS, there is nothing irreplaceable about it, certainly not to warrant any major effort towards preserving it.

I’m not saying that system snapshots are useless (though, it’s my personal opinion that they somewhat are) – I accept that it may be a great convenience to some people to be able to restore a snapshot within minutes. My point is that they are not necessary, because typically there is nothing irreplaceable about what they contain.

What you probably should worry about is the following:

  1. Have multiple external backups of all important files. This includes your documents, pictures, projects you’re working on, any data that is irreplaceable and valuable to you. It does not include the operating system and software that is in the repos. All hardware breaks, especially computers. Hard drives and SSDs fail all the time. Software has bugs that cause data loss. The only effective way to prevent losing important files is to have them backed up. The keywords here are multiple and external. Storage is fairly cheap.

  2. Include a list of installed packages with your backup, as well as important config files. This ensures that in the worst case scenario (e.g. having a boating accident involving your laptop falling in the lake), you can recreate something approximating your system on different hardware in a very short time. Of course, if it is the same hardware, you can just copy the files over, but if it is not, you will have to take into account any hardware differences (some config files might not work on new hardware, some different packages might be needed, etc…), but you can usually work this out manually, instead of just blindly copying over everything.

  3. Have a USB stick with EndeavourOS live image at hand. This is more for convenience than a necessity. Just keep it in a drawer somewhere where you won’t have to search for it. When travelling, put it in your laptop bag. If your bootloader gets messed up, and you don’t have immediate access to another computer, writing an ISO image to a USB drive can be the most difficult part in getting back on your feet.

  4. Read this forum, bookmark it, as well as the Arch wiki. As I’ve already said, if there is any problem affecting many people, chances are it will be known before you know about it, and likely already solved. You’ll save yourself a lot of time by reading this forum. Even if you are the only one affected, asking here for help could save you a lot of frustration. We have one of the best support forums out of any OS, take advantage of that. Also, don’t forget that you can use a live ISO image to browse the internet and read this forum.

  5. Keep your EndeavourOS system updated. Even though bad updates like this one do happen occasionally, this is not a valid reason to delay updating. A rolling release distro is meant to be updated often, and your user experience will be much more pleasant if you do. As long as your important files are backed up, there is nothing to fear when updating. And if your important files are not backed up, data loss is just a matter of time, and probably won’t be related to an update. So back them up, now.

As you can see, only the point no. 1 is actually important. It’s a good idea to follow the other points, too, as they will reduce inconvenience, but certainly not crucial.

Finally, if you are prone to panicking, be aware that a person in a state of panic will often make bad decisions. You can easily make the problem worse if you try to solve it while in such a state. Also, your health is more important than any computer. Always remember that if you have everything that is irreplaceable backed up, any hardware or software failure is nothing more than an inconvenience (and maybe an unpleasant hit to your wallet). A broken bootloader is really nothing to worry about. If your computer can’t boot, relax, drink a big glass of water (important to always be well hydrated :frog:) and proceed to fix it.

9 Likes

I mean, if you wanted btrfs, you will have to, yes.

I like ext4. It’s been a long running staple.

2 Likes

:100:

Sound advice from the :frog: :+1:

Being an idiot, I managed to accidentality delete my .config folder yesterday :person_facepalming:

Because I have a backup to my second drive running every hour, I was able to restore everything immediately :sweat_smile:

1 Like

I was able to repair it to my liking :stuck_out_tongue:

  • rsynced everything of value.
  • wiped the entire disk.
  • reinstalled without issues as btrfs
  • rsynced everything back.

As per anxiety - it’s not much of an issue. I took slow manageable steps towards all my choices.

Thanks for the support folks! :+1:

3 Likes

Funny about that backup and so on… If people are using linux it<s all about to chase problem and install/reinstall many os. This is not mean to be used without any intervention and arch are worst as any update push by random people can cause the system crash. This is all the fun. For now having issues and simply using a pc and have it work all the time : just spin up a vm with no update at all and block all update and all will be fine. That goes for linux stuff centos6 or 7 or windows. Updates are what cause system failure.

Hi, I just noted you mentioned my blog post :smiling_face:

In the meantime, although the AUR package of grub-btrfs has not yet been updated, if you install grub-btrfs from the git repository, they’ve added another service that easily detects timeshift snapshots.

I’ve update my blog post under that respect:

1 Like

First of all: thank you, Mister.

Second then, I think that your update is precious - even if maybe I’m not so experienced as you think.

I’m currently running timeshift 22.11.1-1, but still with the ‘old school’ directory
‘/run/timeshift/backup’
manually created, and the manually added fstab line
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX /run/timeshift/backup btrfs defaults,noatime 0 0 ,
and the ‘sudo systemctl edit --full grub-btrfs.path’ patched with


Requires=run-timeshift-backup.mount

[Path]
PathModified=/run/timeshift/backup/timeshift-btrfs/snapshots

Got to fully understand what/how/if the update you mention changes my current (working, and real-life-tested as working) configuration, and above all if/why I should change it.

But I’m willing to learn, so, once again, thanks!

Peace, Piero