Very slow download speed with pacman and aur

Hi,

I am using Endevour OS since a month, and have been trying to fix the internet speed for pacman and AUR, (using yay). I have tried rankmirrors to find the fastest mirrors, I have tried using reflector for the same, which throws following warnings:

 WARNING: failed to rate http(s) download 
 WARNING: failed to rate rsync download 

but the most I could get out of all this is a bump till 3Mbps. while using the speedtest-cli gets a speed till 60-70Mbps. This also causes the AUR downloads to fail with validity checks, when the AUR packages are above 1GB.

I would be happy to provide any other logs, Any help in this regard would be much appreciated.

I guess it must depend on where you are - although I would have thought that rate-mirrors would have worked OK. For decent speeds it seems the German, US and Canadian (!) mirrors are best bet for out-of-area speeds if your local mirrors are slow. I guess you could post the results of your rate-mirrors run, it might give us some clues…

It seems like you’re failing to connect to the mirrors.

  • Are you from India or around Asia?
  • Can you show us how you use reflector or reflector simple.

Sometimes I also experience this when I use a mobile internet connection with limited bandwidth. In such cases, you may want to change the mirror.

… and to make sure there is no driver issue let’s check hardware information:
for networkdevices:
inxi -Ga

I got this respone:

  Device-1: AMD Renoir vendor: ASUSTeK driver: amdgpu v: kernel
    bus-ID: 05:00.0 chip-ID: 1002:1636 class-ID: 0300
  Device-2: Lenovo FHD Webcam type: USB driver: snd-usb-audio,uvcvideo
    bus-ID: 2-1.4:6 chip-ID: 17ef:4831 class-ID: 0102
    serial: Lenovo FHD Webcam Audio
  Display: x11 server: X.Org 1.21.1.3 driver: loaded: amdgpu display-ID: :0
    screens: 1
  Screen-1: 0 s-res: 3840x1080 s-dpi: 96 s-size: 1016x285mm (40.0x11.2")
    s-diag: 1055mm (41.5")
  Monitor-1: HDMI-A-0 res: 1920x1080 hz: 60 dpi: 92
    size: 528x297mm (20.8x11.7") diag: 606mm (23.9")
  Monitor-2: DisplayPort-1 res: 1920x1080 hz: 60 dpi: 93
    size: 527x296mm (20.7x11.7") diag: 604mm (23.8")
  OpenGL: renderer: AMD RENOIR (DRM 3.42.0 5.15.16-1-lts LLVM 13.0.0)
    v: 4.6 Mesa 21.3.4 direct render: Yes

As I have mentioned, I have changed the mirror a lot of times using the only 2 way I knew how.

I am from India
I am using reflector this way reflector --latest 200 --sort rate --save /etc/pacman.d/mirrorlist

Have you tried lowering the amount of mirrors? I haven’t used reflector in a while now but I think I normally only go 5 or 10 mirrors

1 Like

Please try the application reflector-simple included in EnOS and limit your mirrors to the countries mentioned above known to have updated servers with good speed. And also choose less number of mirrors as mentioned in the previous post.

1 Like

wrong options from my command should be inxi -Na for network devices -Ga is for GPU and displays :wink:

add Germany and USA to be ranked and try using the update mirrors app you can start from the welcome app it shows a GUI with options for countries e.t.c. and show your mirrors list:
cat /etc/pacman.d/mirrorlist | eos-sendlog

I was looking for something network related in the last response. :smile:

  Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
    vendor: ASUSTeK driver: r8169 v: kernel port: fc00 bus-ID: 02:00.0
    chip-ID: 10ec:8168 class-ID: 0200
  Device-2: Intel Wireless 8265 / 8275 driver: iwlwifi v: kernel
    bus-ID: 03:00.0 chip-ID: 8086:24fd class-ID: 0280
  Device-3: Intel Bluetooth wireless interface type: USB driver: btusb
    bus-ID: 6-3:2 chip-ID: 8087:0a2b class-ID: e001

If it helps, mine reflector.conf file:

cat /etc/xdg/reflector/reflector.conf 
# Reflector configuration file for the systemd service.
#
# Empty lines and lines beginning with "#" are ignored.  All other lines should
# contain valid reflector command-line arguments. The lines are parsed with
# Python's shlex modules so standard shell syntax should work. All arguments are
# collected into a single argument list.
#
# See "reflector --help" for details.

# Recommended Options

# Set the output path where the mirrorlist will be saved (--save).
--save /etc/pacman.d/mirrorlist

# Select the transfer protocol (--protocol).
--protocol https

# Select the country (--country).
# Consult the list of available countries with "reflector --list-countries" and
# select the countries nearest to you or the ones that you trust. For example:
# --country France,Germany

# Use only the  most recently synchronized mirrors (--latest).
--latest 5

# Sort the mirrors by synchronization time (--sort).
--sort rate

Also enable systemd reflector timer, which will update your mirrors weekly

sudo systemctl enable reflector.timer

If any time you feel speed is slow, then just run

sudo systemctl start reflector.service
~
❯ cat /etc/pacman.d/mirrorlist | eos-sendlog
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:02:13 --:--:--     0
curl: (28) Failed to connect to clbin.com port 443 after 133286 ms: Connection timed out

This is the error message.

the contnet of my mirrorlist:

################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################
# With:       reflector --verbose -c CA -c DE -c IN -c US --protocol https --sort rate --latest 15 --download-timeout 3
# When:       2022-01-24 12:01:48 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2022-01-24 12:00:22 UTC
# Last Check: 2022-01-24 11:15:27 UTC

## Germany
Server = https://mirror.pkgbuild.com/$repo/os/$arch

## United States
Server = https://arch.mirror.constant.com/$repo/os/$arch

## United States
Server = https://mirror.lty.me/archlinux/$repo/os/$arch

## Germany
Server = https://mirror.f4st.host/archlinux/$repo/os/$arch

## Germany
Server = https://mirror.chaoticum.net/arch/$repo/os/$arch

## United States
Server = https://mirror.hackingand.coffee/arch/$repo/os/$arch

## Germany
Server = https://phinau.de/arch/$repo/os/$arch

## United States
Server = https://america.mirror.pkgbuild.com/$repo/os/$arch

## Canada
Server = https://mirror.csclub.uwaterloo.ca/archlinux/$repo/os/$arch

## Germany
Server = https://mirror.luzea.de/archlinux/$repo/os/$arch

## Germany
Server = https://archlinux.thaller.ws/$repo/os/$arch

## Germany
Server = https://mirror.pseudoform.org/$repo/os/$arch

## Germany
Server = https://de.arch.mirror.kescher.at/$repo/os/$arch

## Germany
Server = https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch

## United States
Server = https://mirror.theash.xyz/arch/$repo/os/$arch

## Worldwide
Server = https://mirror.rackspace.com/archlinux/$repo/os/$arch

if you are using ethernet (wired) it could be that your device is causing slow network issue…

2 Likes

Tried a combination of all the helpful suggestions, which includes using wireless instead of LAN. but the results still remain the same.

you should try also rate-mirrors it has some smart stuff that also counts in real-world infrastructural stuff like undersea cables and other stuff related to your location.

 » rate-mirrors --help
rate-mirrors config 0.9.0
Usually default options should work

USAGE:
    rate-mirrors [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
        --allow-root    
            allow running by root

    -h, --help          
            Prints help information

    -V, --version       
            Prints version information


OPTIONS:
        --concurrency <concurrency>
            Number of simultaneous speed tests [default: 8]

        --country-neighbors-per-country <country-neighbors-per-country>
            Neighbor country to test per country [default: 3]

        --country-test-mirrors-per-country <country-test-mirrors-per-country>
            Number of mirrors to test per country [default: 2]

        --entry-country <entry-country>
            Entry country - first country (+ its neighbours) to test.
            You don't need to change it unless you are just curious. [default: US]
        --eps <eps>
            Per-mirror: sigma to mean speed ratio
            
              1.0 -- 68% probability (1 sigma), no 100% error
              0.5 -- 68% probability (1 sigma), no 50% error;
              0.25 -- 68% probability (1 sigma), no 25% error;
              0.125 -- 95% probability (2 sigmas), no 25% error;
              0.0625 -- 95% probability (2 sigmas), no 12.5% error: [default: 0.0625]
        --eps-checks <eps-checks>
            Per-mirror: after min measurement time elapsed, check such number of subsequently downloaded data chunks
            whether speed variations are less then "eps" [default: 40]
        --max-jumps <max-jumps>
            Max number of jumps between countries, when finding top mirrors [default: 7]

        --min-bytes-per-mirror <min-bytes-per-mirror>
            Minimum number of bytes to be downloaded, required to measure mirror speed [default: 70000]

        --min-per-mirror <min-per-mirror>
            Minimum downloading time, required to measure mirror speed, in milliseconds [default: 300]

        --per-mirror-timeout <per-mirror-timeout>
            Per-mirror speed test timeout in milliseconds [default: 1500]

        --protocol <protocol>...
            Test only specified protocols (can be passed multiple times)

        --save <save-to-file>
            Filename to save the output to in case of success

        --top-mirrors-number-to-retest <top-mirrors-number-to-retest>
            Number of top mirrors to retest [default: 5]


SUBCOMMANDS:
    arch           fetch & test archlinux mirrors
    artix          fetch & test artix mirrors
    cachyos        fetch & test cachyos mirrors
    endeavouros    fetch & test endeavouros mirrors
    help           Prints this message or the help of the given subcommand(s)
    manjaro        fetch & test manjaro mirrors
    rebornos       fetch & test rebornos mirrors
    stdin          accepts lines of urls OR lines with tab-separated urls and countries

do you got other/different results on checking internet speed with wifi?

This is a sign that your issue is not really related to Arch mirrors and your mirrorlist selection, since AUR packages are not related to that. It’s a network problem.

A usual test for download speeds (disconnected from random speed-test providers) is to test downloading (temporarily, delete after) a Linux distro ISO from different servers, for example an archlinux ISO from random worldwide servers. Use wget or uGet (GUI) to have a cleaner outcome.

This may give you one clue about the real issue.

I think I have heard people from India complaining about the same thing, so maybe it has to do with internet provider or country standards/complications(?).

1 Like