I’m fairly new to Arch, but I’ve been doing IT for like… 15+ years.
With the whole Redhat Drama, It got me thinking why I havn’t seen any Arch-based server distros.
With CICD tools, it would seem like an ideal candidate for a server/container, especially for Green/Blue deployments… so why haven’t we seen them?
I do not admin a server, so take this with a grain of salt. Isn’t one of the points of a server to provide reliable and sustained uptime? If so, why would anyone want a server based on a rolling release distro? It just seems counter-intuitive to me.
Sort of… the definition of “server” has morphed considerably in the last decade alone. Today, a server hosts a service, and we treat them like cattle, not pets.
What that means is we’ve shifted from these monolithic “better not touch that box!” Servers that can only be restarted once a month to containers, where the thought process is “Oh, it’s having problems? Terminate it and rebuild”. Even with full blown VMs like AWS EC2, we terminate and rebuild. Services like AWS ECS and Kubernetes will actively kill off unhealthy containers and automatically rebuild new ones to replace them.
So how do we integrate reliability in an environment where containers/servers are constantly getting killed and rebuilt? We don’t focus on the servers, we focus on the process - We focus on creating consistent, repeatable builds. And that’s where tools like terraform and Ansible really shine.
So, why not Arch? You don’t ever really have to “upgrade” containers, you pull a new image and create a new container. If you’re keeping servers around indefinitely, like, 6 months +, your build process isn’t consistent, or it’s legacy and is awaiting refactoring, or it’s an actual physical server, in which case, it should have an LTS OS on it. But that’s an edge case.
So, help me understand that the hardware and operating system that runs the containers does not still require some degree of reliability and predictability? I think I understand that you are saying the server is just a container, but that container needs to run on some real server, located somewhere and running some operating system, right? I, as the client, may only interact with the services being provided by a container, but that container needs a stable platform on which to run, does it not?
I am also not understanding how consistent and repeatable builds are accomplished on a system that is continually updated, like Arch Linux. If all the system libraries are in a constant state of flux, if all the package versions of things like MariaDB, Glibc, Python, SSH, etc. and the countless other components, are constantly being updated, how can consistent and repeatable builds of containers take place?
Also, what about security updates? Rolling release distros do not get security updates, not in the strict sense. Rolling release distros get package updates, which may address certain security vulnerabilities, but are just as likely to increase the number of security problems as they attempt to solve.
Basically just much more hassle for average maintainer while configuring and running, but if i’d choose to run rolling release server for some reason - that would be something like Alpine Linux.
You use CICD tools on your private server, not for your company with many employees. Right?
If you want to maintain, refresh and reboot the server often per day or per week, Arch server would suit you.
I also use Arch server at my home. Our company uses Debian server without reboot, which should run 24 hours a day, but maybe 1 - 2 time reboot per year after upgrade.
I think it mainly matters what you using the server for. For instance, I run EnOS as a server for my repo hosting (EndeavourOS and chaotic-AUR) - mostly because I don’t have to renew any knowledge of alternative OS’s that I have left behind. Stability is absolutely not a problem, although I do have a ‘plan’ in place to avoid potential problems (which haven’t happened in a long time, BTW). Basically, I run updates once a week - but only up to a date one week prior to current. I have found over the years hat a week is sufficient for someone to have found and fixed whatever comes up. Even if I somehow miss the ‘problem’, my chances are good that it has been ‘automagically’ caught with my delay trategy - but I would expect to know about it (and any workarounds) before updating. Also - should I have to rebuild it, my work would be limited to about 20 minutes to restart it all.
Go with what works for you - all the experts out there can only tell you what works for them
The biggest issue with running Arch as a server in my experience is library instability. Since the libraries are always changing, so does comparability with whatever you are running on the server. This may or may not be an issue.
For example, if you are running a DHCP server using bind, this is probably going to be fine.
If you are using something that requires a certain version of php or pre-packaged binaries that rely on certain libraries you will have a bad time.
For everything else, it just depends. I used to run Arch-based servers but it just isn’t worth the hassle.
Chaotic libraries would make development hell, let alone maintenance. That would be a hard line for most, if not all, enterprises, to cross, essentially Defeating any demand for an Arch-based server.
The base, bare metal operating system on a server can be anything from a specialized hypervisor OS, like VMware esxi or an Ubuntu server running KVM/Qemu or something more complicated like openstack or kubernetes. They can also host into the thousands of containers/vm’s depending on configuration.
On those servers, you absolutely would NOT want Arch. But, those Host operating systems are often run pretty bare in order to provide as much of the CPU/Ram to the virtualized servers, reduce attach surface, etc. So yes, they are reliable monoliths, but they don’t handle data-processing and have the capability to be replaced/reconfigured without causing downtime. Essentially, they would be like nodes in a cluster.
The second and third part of your response are what I was missing. I didn’t realize the libraries were changing that often, I just thought new versions were constantly being pushed. I also just assumed that with the updates there were also more security patches. But it makes sense that an arch-based server would basically be running headlong into zero-days and such that could easily be avoided by not pulling the most recent packages.
Not directly, but Arch provides for the need. EndeavourOS has a little utility (eos-shiftttime) that automates/eases the task with a bit of help from yad - it is in the repos.