Install fails due to consistent 404/403s from mirrors

I’m trying to install in a parallels VM following this tutorial on YouTube:

Interestingly in the first case (using the global mirror), while the first wave of pacman fetches fails out fast, the installer continues and the second wave succeeds. This results in a broken install that exactly matches this post: Pinebook Pro: Lightdm fails to start on first boot. "Can't launch X server X, not found in path"

pacman -S xorg-server fixes it for me as well… Firefox is missing of course, but if I switch the mirror list to somewhere else, syncing the package succeeds and Firefox installs fine after that.

My network setup is pretty vanilla, no VPNs, unusual firewall rules, proxies, etc. I’ve installed non-ARM endeavour just fine without any issues syncing packages.

@cowsay @Shjim you both might find this interesting :slight_smile:

1 Like

This is a redirector service so pushes requests to various mirrors in the pool - some may be out-of-sync, hence occasional issues downloading packages.

Sounds like an incorrectly-configured mirror server, or it has blocked you for some reason. Check you can load it in a browser (I can).

1 Like

Just as note: I cannot load this directly - “DNS address could not be found”. This suggests the problem may be widespread…

Note: not using this myself, just checking!

work ok for me

Did some digging, the 404s are coming from http://il.us.mirror.archlinuxarm.org

Broken: http://il.us.mirror.archlinuxarm.org/aarch64/extra/firefox-92.0.1-1.1-aarch64.pkg.tar.xz
Not Broken: http://ca.us.mirror.archlinuxarm.org/aarch64/extra/firefox-92.0.1-1.1-aarch64.pkg.tar.xz

The file is conspicuously absent in this index: http://il.us.mirror.archlinuxarm.org/aarch64/extra

May look into the 403s coming from the ca mirror later, I’m curious :slight_smile:

I think the 403s on the ca mirror are due some form of rate limiting. After the installer fails, I can wget the package and install it with pacman. For posterity the sequence of events are:

Base Addons:
fetches ~160 packages successfully
rtmpdump: 403 error
libdvdread: 403 error
libdca: 403 error
rp-pppoe: 403 error

[Does other things]

Installing i3-wm:
Syncs ~105 packages successfully
clutter-gtk: 403 error
libcddb: 403 error

I can successfully sync these packages manually after the install completes.

The instructions recommend changing ParallelDownloads: “Uncomment this line by removing the leading # and if desired, change 5 to 8 or however many parallel downloads you want to run. For my ISP 8 works well.”
https://arm.endeavouros.com/endeavouros-arm-install/

I commented out the line (so disable parallel downloading) and was able to fully complete the install on the CA mirror with no errors.

So in summary:

  • il.us.mirror.archlinuxarm.org appears to be missing packages (now 12 hours from when I first tried)
    • if this is normal, the installer should probably try other mirrors, or at least fail more loudly. If someone walks away during the install and comes back, the installer will appear to have completed normally but they will end up with a broken install
    • Last modified date on IL mirror for extra.db is 2021-09-25 11:26, the Firefox package on the CA mirror is 2021-09-24 22:54. There are other packages that have a similar last modified date on the CA mirror that are missing in the IL mirror. I don’t know enough about the package system to know if this is bad or not, but seems worth mentioning. (this may be an upstream thing)
  • ca.us.mirror.archlinuxarm.org is rate limiting

I’m not sure there’s much reason to manually adjust this, pacman will adjust the number of concurrent downloads depending on download speeds (e.g. at work it will run five or six concurrently, at home it sticks to one).

I marked my previous comment as a solution. For what it’s worth, it looks like the IL mirror has the missing files now, and I get a clean install out of using it now.

Just a suggestion… if it’s normal for a mirror to get into a broken state (it was in this state for at least 12 hours), it would be more helpful if the installer bailed out with a clear error, rather than giving the appearance of success but leaving the machine with subtle (or not subtle) problems. Or even better, be robust enough to try another mirror.

Also the only reason I changed ParallelDownloads is because that’s what the instructions said to do… maybe it’s worth mentioning there when suggesting to do this that some mirrors won’t play nice with this option. From the man page (https://archlinux.org/pacman/pacman.conf.5.html), by default (left commented out) it would just use a single download stream at a time. As far as pacman adjusting the number of concurrent downloads, I don’t think a client like pacman would know that it’s being rate-limited by the server from 403 error codes. (It probably should have been returning a 429 instead).

(p.s. thanks to everyone who has worked on this project… aside from some bumps with syncing down packages when using ARM, the first experience with EndeavourOS has been nice!)

1 Like

@aclysma
I am in Colorado, and I get the best results from the IL and FL (Illinois and Florida) mirrors.

Also, if you use a vpn service, disabling it helps greatly.

Pudge

Install Arch /EndeavourOS VM on M1 Mac using parallels. Quite interesting.

Out of curiosity, how is this working for you.

Pudge

2 Likes

I’ve only had it up and running for a few hours and I’m very much a novice with desktop linux (Some familiarity with “server” linux, usually I’m working with teams that have dedicated ops/network engineers. This is a learning experience :slight_smile:). So I’m probably not the most qualified to say, but so far I haven’t noticed anything odd.

I think for anyone wanting to try out linux on arm (maybe to test/improve it), or to have it run some services locally like a DB server while doing development in macOS, it would be fine. (Although they may have a smoother experience on a distro that parallels “officially” supports.) I was able to install/run their guest agent just fine in arch and endeavour.

2 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.