How to setup reflector-auto?

Hello everyone! This is my first post here and great distro to learn arch and all the tools from arch.
I migrated few days ago from ubuntu 19.10 because my late-late upgrade to 20.04 LTS broke my linux-installation. I have once installed arch to virtual box (very minimal and hard to use) so i have little basic understanding.Still newbie.

Perhaps offtopic but because dev said that reflecor-auto is not supported until end of year I tried and tried to understand reflector from arch repos. Man pages were first very confusing and hard to understand. I finally used these options in my

/etc/xdg/reflector/reflector.conf

–save /etc/pacman.d/mirrorlist
–protocol https
–country FI,SE,NO,DK,DE,NL
–sort rate
–age 4
–completion-percent 100
–score 30

Lot of options but my country doesnt have so many servers around. But these options dont conflict with each other badly so when i launch in terminal

sudo systemctl reflector.service

this runs command reflector with my conf. file options and updates my mirrorlist. Can be used anytime.

If I want the mirrorlist updated weekly then i start the timer with

sudo systemctl enable reflector.timer

sudo systemctl start reflector.timer

systemctl status reflector.timer

Great program. My mirrors with these options are on top of the arch linux mirrorlist rating. Hope this clarified something :grinning:

3 Likes

Just came across this thread today! Normally I just open up reflector-simple every so often (maybe like 1-2 times a month?) to update the mirrorlist. If I can set it and forget it with reflector-auto, then I’m all for trying that out. Now I haven’t done anything explicitly yet since I would like to make sure what I’m doing is verified by someone else first before I go ahead and save everything. On my system I already have reflector and reflector-simplepackages installed. I’ve used the following command:

sudo nano /etc/xdg/reflector/reflector.conf

and my edited file looks like the following below. Note, I am in the US, so if anyone can just make sure I did this all correctly, I’ll go ahead and save it and then run sudo systemctl enable --now reflector.timer to allow this to run each week so I don’t have to worry or think about it anymore :slight_smile:

  GNU nano 5.8                                                                   /etc/xdg/reflector/reflector.conf                                                                             
#
# See "reflector --help" for details.

# Recommended Options

# Set the output path where the mirrorlist will be saved (--save).
--save /etc/pacman.d/mirrorlist

# Select the transfer protocol (--protocol).
--protocol https

# Select the country (--country).
# Consult the list of available countries with "reflector --list-countries" and
# select the countries nearest to you or the ones that you trust. For example:
--country US

# Use only the  most recently synchronized mirrors (--latest).
--latest 10 (default was 5, I changed it to 10, but should it be 20 maybe?--this message will be deleted before I save this file, fyi--)

# Sort the mirrors by synchronization time (--sort).
--sort age

Basically I just want to do what reflector-simple does, but automatically. The comment above me @blahnik added a few other values, i.e. sort rate, age 4, completion-percent 100, score 30, and I’m wondering also if I should or need to include any of those? Any additional help or pointing me in the right direction would be most appreciated, thank you!

Edit: Side note, I know this topic is from last year, so I don’t know at all if it still applies and is relevant now or if maybe I should just ignore it all, please let me know, thanks.

I don’t bother with automation - and reflector has auto capability itself now (if you want it) so reflector-auto has drifted away…

Just man reflector I guess! (and follow along with SYSTEMD INTEGRATION entries)

I just ranked US mirrors (note: from Europe!) and here are the results:

## United States
Mirror-url                                                      Age(sec) Fetch-time(sec)
https://america.mirror.pkgbuild.com/$repo/os/$arch              127      0.482402
https://arch.mirror.constant.com/$repo/os/$arch                 127      0.491412
https://arlm.tyzoid.com/$repo/os/$arch                          127      0.559468
https://mirror.hackingand.coffee/arch/$repo/os/$arch            127      0.685778
https://mirror.lty.me/archlinux/$repo/os/$arch                  127      0.688026
https://mirrors.lug.mtu.edu/archlinux/$repo/os/$arch            127      1.007911
https://iad.mirrors.misaka.one/archlinux/$repo/os/$arch         6643     0.413812
https://mirrors.mit.edu/archlinux/$repo/os/$arch                6643     0.496757
https://mirror.stephen304.com/archlinux/$repo/os/$arch          6643     0.518642
https://mirror.wdc1.us.leaseweb.net/archlinux/$repo/os/$arch    6643     0.561329
https://mirrors.xtom.com/archlinux/$repo/os/$arch               6643     0.592933
https://ftp.sudhip.com/archlinux/$repo/os/$arch                 6643     0.612163
https://arch.hu.fo/archlinux/$repo/os/$arch                     6643     0.613171
https://archmirror1.octyl.net/$repo/os/$arch                    6643     0.617950
https://mirror.mia11.us.leaseweb.net/archlinux/$repo/os/$arch   6643     0.643332
https://zxcvfdsa.com/arch/$repo/os/$arch                        6643     0.666055
https://mirror.dal10.us.leaseweb.net/archlinux/$repo/os/$arch   6643     0.674877
https://iad.mirror.rackspace.com/archlinux/$repo/os/$arch       6643     0.702615
https://plug-mirror.rcac.purdue.edu/archlinux/$repo/os/$arch    6643     0.759964
https://mirror.phx1.us.spryservers.net/archlinux/$repo/os/$arch 6643     0.764215
https://mirrors.rit.edu/archlinux/$repo/os/$arch                6643     0.800443
https://mirror.sfo12.us.leaseweb.net/archlinux/$repo/os/$arch   6643     0.805627
https://dfw.mirror.rackspace.com/archlinux/$repo/os/$arch       6643     0.853902
https://mirror.arizona.edu/archlinux/$repo/os/$arch             6643     0.874193
https://ord.mirror.rackspace.com/archlinux/$repo/os/$arch       6643     0.874807
https://repo.ialab.dsu.edu/archlinux/$repo/os/$arch             6643     0.982971
https://mirror.kaminski.io/archlinux/$repo/os/$arch             6643     1.117354
https://mirrors.ocf.berkeley.edu/archlinux/$repo/os/$arch       6643     6.429470
https://mirrors.sonic.net/archlinux/$repo/os/$arch              12251    0.968032
https://mirror.ette.biz/archlinux/$repo/os/$arch                16149    0.521209
https://arch.mirror.square-r00t.net/$repo/os/$arch              65154    0.520248
https://mirror.pit.teraswitch.com/archlinux/$repo/os/$arch      154904   0.412212
https://mirrors.rutgers.edu/archlinux/$repo/os/$arch            11816622 0.938160

Please note the numbers on the right:

  • Age means how recently the mirror has been updated (smaller is better)
  • Fetch-time measures the speed (smaller is better)

This list shows that some mirrors are well updated, and some are not.

After this introduction, I’d like to suggest the options to use:

--save /etc/pacman.d/mirrorlist
--protocol https
--latest 5
--sort rate
--country US

You can of course try that with command:

reflector --protocol https --latest 5 --sort rate --country US

and see the result (it doesn’t change any file). And please compare your result with the list above!

Note that mirrors will be updated constantly but not at the same time. This will create variations in ranking.
That’s why I wouldn’t recommend a bigger number for --latest.
Adding --age 2 looks like being a good idea too, but it may also cause an empty mirrorlist…
So this is my suggestion and some reasoning. Feel free to disagree and change values and options. As said, I’m not ranking in the US, so your results might be different.

1 Like

When I try your command:

reflector --protocol https --latest 5 --sort rate --country US

This is the following output I get so far (not sure how you got the Age/Fetch-time to show):

[scott@endeavourOS ~]$ reflector --protocol https --latest 5 --sort rate --country US
################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################

# With:       reflector --protocol https --latest 5 --sort rate --country US
# When:       2021-08-03 20:30:08 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2021-08-03 20:29:47 UTC
# Last Check: 2021-08-03 19:11:47 UTC

Server = https://arlm.tyzoid.com/$repo/os/$arch
Server = https://arch.mirror.square-r00t.net/$repo/os/$arch
Server = https://america.mirror.pkgbuild.com/$repo/os/$arch
Server = https://mirror.lty.me/archlinux/$repo/os/$arch
Server = https://arch.mirror.constant.com/$repo/os/$arch

Reflector doesn’t show them, I used another program.

The list looks nice except arch.mirror.square-r00t.net which is not recently updated.

It would be interesting to see if you add option --age 2. How does it change the list?

1 Like

Ask and you shall receive! Edit: (looks like arch.mirror.square-r00t.net is still in there)

[scott@endeavourOS ~]$ reflector --protocol https --latest 5 --sort rate --country US --age 2
################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################

# With:       reflector --protocol https --latest 5 --sort rate --country US --age 2
# When:       2021-08-03 20:48:33 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2021-08-03 20:47:26 UTC
# Last Check: 2021-08-03 20:43:15 UTC

Server = https://arch.mirror.square-r00t.net/$repo/os/$arch
Server = https://mirror.hackingand.coffee/arch/$repo/os/$arch
Server = https://mirror.lty.me/archlinux/$repo/os/$arch
Server = https://america.mirror.pkgbuild.com/$repo/os/$arch
Server = https://arch.mirror.constant.com/$repo/os/$arch


Strange. it may be that the other program I use is too experimental… :wink:

So it looks like I’m at a stalemate then eh? :thinking:

Maybe…
I actually do not know US mirrors that much, the only information I have is by using these interesting little programs.

Anyway, your list looks good except the square-r00t thingy.
But if needed, that can be removed by a script after running reflector.
So with small tricks it can be made work a bit better. :smile:

When I check out https://archlinux.org/mirrors/status/ and I search for the " arch.mirror.square-r00t.net" mirror which you mentioned wasn’t recently updated, but according to the mirror status link from above it’s at 100% or am I not understanding any of this? :stuck_out_tongue:

@manuel I don’t know if maybe this could shed some light on some things, but this is what my current /etc/pacman.d/mirrorlist file looks like:

  GNU nano 5.8                                                                       /etc/pacman.d/mirrorlist                                                                                  
################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################
# With:       reflector --verbose -c US --protocol https --sort rate --latest 10
# When:       2021-07-31 19:58:57 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2021-07-31 19:57:54 UTC
# Last Check: 2021-07-31 18:46:08 UTC

## United States
Server = https://mirror.hackingand.coffee/arch/$repo/os/$arch

## United States
Server = https://mirror.lty.me/archlinux/$repo/os/$arch

## United States
Server = https://arlm.tyzoid.com/$repo/os/$arch

## United States
Server = https://mirrors.rit.edu/archlinux/$repo/os/$arch

## United States
Server = https://mirrors.lug.mtu.edu/archlinux/$repo/os/$arch

## United States
Server = https://archmirror1.octyl.net/$repo/os/$arch

## United States
Server = https://america.mirror.pkgbuild.com/$repo/os/$arch

## United States
Server = https://mirrors.xtom.com/archlinux/$repo/os/$arch

## United States
Server = https://arch.mirror.constant.com/$repo/os/$arch

## United States
Server = https://mirrors.ocf.berkeley.edu/archlinux/$repo/os/$arch

Edit: 9 out of 10 of those mirrors look good at 100% completion as shown on https://archlinux.org/mirrors/status/
The only mirror that was at 91% was the mirrors.xtom.com/archlinux/
Not sure if I should maybe just re-run reflector-simple since this file was created on 2021-07-31 to perhaps refresh this mirror list. I may be looking into this way too much :stuck_out_tongue:

Edit 2: is there a way to auto run reflector-simple once a week?

I only got this.

[ricklinux@eos-kde ~]$ reflector --protocol https --latest 5 --sort rate --country US
################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################

# With:       reflector --protocol https --latest 5 --sort rate --country US
# When:       2021-08-03 22:13:28 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2021-08-03 22:13:24 UTC
# Last Check: 2021-08-03 20:43:15 UTC

Server = https://arch.mirror.square-r00t.net/$repo/os/$arch
Server = https://arch.mirror.constant.com/$repo/os/$arch
Server = https://america.mirror.pkgbuild.com/$repo/os/$arch
Server = https://mirror.lty.me/archlinux/$repo/os/$arch
Server = https://mirror.hackingand.coffee/arch/$repo/os/$arch

[ricklinux@eos-kde ~]$ 

Go here: https://archlinux.org/mirrorlist/

Back up your mirror list and replace with what was generated. Use only HTTPS mirrors.

2 Likes

Not sure what I did, but there were a few package updates, so I ran yay -Syu and got the following errors, any thoughts?

[scott@endeavourOS ~]$ yay -Syu
[sudo] password for scott: 
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
 endeavouros is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Package (5)           Old Version   New Version   Net Change  Download Size

community/gtkd        3.9.0-12      3.9.0-13       -0.51 MiB       4.48 MiB
community/hwinfo      21.75-1       21.76-1         0.00 MiB       0.96 MiB
community/liblphobos  2:1.26.0-4    2:1.27.0-1      3.54 MiB      12.28 MiB
extra/re2             1:20210601-1  1:20210801-1    0.00 MiB       0.17 MiB
community/tilix       1.9.4-3       1.9.4-4         0.07 MiB       0.92 MiB

Total Download Size:    18.82 MiB
Total Installed Size:  129.60 MiB
Net Upgrade Size:        3.11 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
 re2-1:20210801-1...   173.1 KiB  30.3 KiB/s 00:06 [-----------------------] 100%
 gtkd-3.9.0-13-x86_64    4.5 MiB   790 KiB/s 00:06 [-----------------------] 100%
 tilix-1.9.4-4-x86_64  946.3 KiB  97.5 KiB/s 00:10 [-----------------------] 100%
 liblphobos-2:1.2...    12.3 MiB  1204 KiB/s 00:10 [-----------------------] 100%
 hwinfo-21.76-1-x...   985.8 KiB  92.3 KiB/s 00:11 [-----------------------] 100%
 Total (5/5)            18.8 MiB  1775 KiB/s 00:11 [-----------------------] 100%
error: failed retrieving file 'liblphobos-2:1.27.0-1-x86_64.pkg.tar.zst' from arch.mirror.square-r00t.net : The requested URL returned error: 404
error: failed retrieving file 'gtkd-3.9.0-13-x86_64.pkg.tar.zst' from arch.mirror.square-r00t.net : The requested URL returned error: 404
error: failed retrieving file 'hwinfo-21.76-1-x86_64.pkg.tar.zst' from arch.mirror.square-r00t.net : The requested URL returned error: 404
warning: too many errors from arch.mirror.square-r00t.net, skipping for the remainder of this transaction
error: failed retrieving file 're2-1:20210801-1-x86_64.pkg.tar.zst' from arch.mirror.square-r00t.net : The requested URL returned error: 404
error: failed retrieving file 'tilix-1.9.4-4-x86_64.pkg.tar.zst' from arch.mirror.square-r00t.net : The requested URL returned error: 404
(5/5) checking keys in keyring                     [-----------------------] 100%
(5/5) checking package integrity                   [-----------------------] 100%
(5/5) loading package files                        [-----------------------] 100%
(5/5) checking for file conflicts                  [-----------------------] 100%
(5/5) checking available disk space                [-----------------------] 100%
:: Processing package changes...
(1/5) upgrading liblphobos                         [-----------------------] 100%
(2/5) upgrading gtkd                               [-----------------------] 100%
(3/5) upgrading hwinfo                             [-----------------------] 100%
(4/5) upgrading re2                                [-----------------------] 100%
(5/5) upgrading tilix                              [-----------------------] 100%
:: Running post-transaction hooks...
(1/4) Arming ConditionNeedsUpdate...
(2/4) Compiling GSettings XML schema files...
(3/4) Updating icon theme caches...
(4/4) Updating the desktop file MIME type cache...
:: Searching databases for updates...
:: Searching AUR for updates...
 there is nothing to do

I run this ~3 months or whenever I remember. Don’t forget to -Syyu when done. No need for anything more, and if I forget the command (I always do - I just check /etc/pacman.d/mirrorlist and copy/paste). No need to setup something that runs hourly/daily whatever IMO. I mean, I guess I’ve just never had that much of an issue. Maybe other parts of the world mirrors go down regularly, but here in Southern California USA, it’s not really an issue.

reflector --protocol https --verbose --latest 25 --sort rate --save /etc/pacman.d/mirrorlist

I also think that in most cases it is not very useful to re-create the mirrorlist too often.

Instead you could examine the results for some days and manually remove mirrors that cause trouble.

For example, I’d remove arch.mirror.square-r00t.net permanently from the list.

2 Likes

That just as simple as sudo nano /etc/pacman.d/mirrorlist and deleting that server from the list and saving it I take it?

1 Like

Exactly. Very easy.

1 Like

I prefer Kate, but that’s it.