Updates: Geofencing and local need to restrict endeavouros-mirrorlist

[julia-n]

Good day,

I have a local need to restrict the automatic update of endeavouros-mirrorlist: Normally I download the latest mirrorlist separately, every week or so, and manually inspect and edit out specific entries, and save, and then run sudo pacman -Syu, usually followed by yay -Syu and yay -Scc. Today, after starting pacman, I noticed that endeavouros/endeavouros-mirrorlist is being included near the top of the package list (old ver: 4.11.1, new ver: 4.12-1).

I update almost daily. I would rather that there was just a notification of my version vs any new version, so I could simply decide whether or not to abort the installation, and manually update the mirrors, or continue without any automatic overwrite of my version of endeavouros-mirrorlist. I need to know which mirrors are changing, and be able to restrict the list, before any updates are made.

I thought this automatic update of mirrors was only available when you execute sudo pacman -Syy, not -Syu.

I have read through Wiki entries on pacman and mirrors, and might have missed something. I thought maybe I could download the mirrorlist from the update button on the Welcome App first, and then maybe the pacman -Syu wouldn’t include it, but wasn’t sure if there was a better way. How can I best disable this automatic update of the endeavouros-mirrorlist? Are there other solutions? Thank you.

[BluishHumility]

When there is an update to the mirrorlist, it is saved as a .pacnew file (check: /etc/pacman.d/endeavouros-mirrorlist.pacnew). At your convenience, you may review the new mirrorlist and merge any changes you wish to adopt into your mirrorlist file with meld or whatever diff program you prefer.

[dalto]

The mirrorlist update shouldn’t overwrite your mirrorlist if you have modified it. It should be safe to let the package update if I am understanding what you are trying to accomplish. As @BluishHumility points out, it will create a pacnew file which you can decide how to handle when you are ready.

[manuel]

There’s the utility checkupdates which will tell you which updates are available.

Also, for AUR packages, you may use command yay -Qua for checking what the updates are.

In addition, to keep specific EndeavourOS mirrors as first during ranking, you can set ALWAYS_FIRST_EOS_MIRRORS in file /etc/eos-rankmirrors.conf to list the preferred mirrors in the order you want.
Probably you want to see the value of EOS_AUTO_MIRROR_RANKING as well.

And, as mentioned above, eos-rankmirrors should create a .pacnew file as the ranking result. You can use eos-pacdiff to see and possibly merge the changes of your choice.

[julia-n]

Thank you everybody, for the excellent and trusted explanations.
(And I now understand the “warning: /etc/pacman.d/endeavouros-mirrorlist installed as /etc/pacman.d/endeavouros-mirrorlist.pacnew”)

[julia-n]

I have been mulling over this issue for awhile now, in between work, having emergency surgery, and adopting my first puppy (yikes!).

I don’t think that the update eos mirrors and ranking should be highly coupled with the sudo pacman -Syu and yay -Syu workflows: mirrors should be independent and optional, as a separate mirror maintenance workflow. (This doesn’t mean there can’t be a separate simple validity check of the mirrors before the update workflows are called.)

For example, I like to update the rolling arch core almost everyday I use it on multiple machines and multiboot configurations, and eos separately, and as I build upon my development pipeline, I plan on using eos for major parts of my continuous integration and deployment pipelines, which will likely include Spack (https://spack.io/).

I like to update the mirrors much less frequently, and find when there is a high frequency of re-ranking (even in the matter of a few hours, the ranks change up and down radically, and I won’t even use half the provided mirrors anyway). It seems somewhat silly to rerank so frequently, especially since there is about a 50/50 chance that one or more of the mirrors will timeout, adding a lot of unnecessary time to the sudo pacman -Syu operation. That, and all it is doing is writing an endeavouros-mirrorlist.pacnew over and over, on each machine I run. I won’t even use these pacnew files, as I fetch, review, and edit my own endeavourous-mirrorlist, once, and I fan out the same result to each target machine.

If the EndeavourOS team doesn’t mind, I think there needs to be a complete code review of the parts where the: ./eos-rankmirrors, ./eos-rankmirrors-from-hook, and related friends are coupled with the pacman call. Additionally, there are hard-coded URLs in these shell scripts, which should really be provided through a .conf file, and at the very least, there should be an option to skip this entirely, if tightly coupled as it currently is.

Please consider it, and let me know what you think. I’m willing to help, I really like EndeavourOS, but this little papercut causes it to be slightly less fun, and I think fixing the issue would add some polish.

Thank you, Have a nice week!

[dalto]

@julia-n

I don’t think that the update eos mirrors and ranking should be highly coupled with the sudo pacman -Syu and yay -Syu workflows: mirrors should be independent and optional, as a separate mirror maintenance workflow. (This doesn’t mean there can’t be a separate simple validity check of the mirrors before the update workflows are called.)

That is how it is by default. Personally, I only rank my mirrors 1-2 times per year.

@julia-n

I like to update the mirrors much less frequently, and find when there is a high frequency of re-ranking (even in the matter of a few hours, the ranks change up and down radically, and I won’t even use half the provided mirrors anyway). It seems somewhat silly to rerank so frequently, especially since there is about a 50/50 chance that one or more of the mirrors will timeout, adding a lot of unnecessary time to the sudo pacman -Syu operation. That, and all it is doing is writing an endeavouros-mirrorlist.pacnew over and over, on each machine I run. I won’t even use these pacnew files, as I fetch, review, and edit my own endeavourous-mirrorlist, once, and I fan out the same result to each target machine.

The hook only runs when the mirrorlist package changes. Basically, when we add or remove mirrors.

@julia-n

at the very least, there should be an option to skip this entirely,

You can skip it entirely. pacman/alpm has this functionality built-in. You can inhibit the hook by putting an empty file or a symlink to /dev/null at /etc/pacman.d/hooks/eos-rankmirrors.hook

Alternatively, you can uninstall the package eos-rankmirrors and pick a mirror manually.

[julia-n]

Thank you, dalto.

@dalto

The hook only runs when the mirrorlist package changes. Basically, when we add or remove mirrors.

Perhaps I made a mistake in my maintenance of old packages. I haven’t run sudo pacman -Syu in the past 3 days, as I relaxed my updates until the Arch updates migration to git was completed, but in the many days before that, I was getting the automated re-ranking of mirrors saved to the /etc/pacman.d/endeavouros-mirrorlist.pacnew every single time I ran pacman -Syu, sometimes multiple times in the same day. The “broken-record” issue with re-ranking did not appear this morning with my first update post migration.

@dalto

You can skip it entirely. pacman/alpm has this functionality built-in. You can inhibit the hook by putting an empty file or a symlink to /dev/null at /etc/pacman.d/hooks/eos-rankmirrors.hook

Alternatively, you can uninstall the package eos-rankmirrors and pick a mirror manually.

Thank you, I will watch the issue closely, and if it continues to interfere, I will implement these steps, as this is the answer to the simple question I should have actually asked in the first place: “How to disable automatic re-ranking of eos mirrors.” All part of my learning process.