Bad experience with systemd-resolved

on a spare pc, i’ve been using artix dinit since the start of the year, blindingly fast and not a single hung process to date, very easy to administer to dinit too. Can’t fault it so far.

1 Like

Thanks for sharing!
I am being very tempted to try out some of the distros on the list I posted above.
Atrix is a very compelling candidate. Surely I will face a learning curve for doing things differently than how it is done with systemd but good to know that it is easy to administer. Thanks!

1 Like

if you do and get stumped on something, feel free to send me a personal message.

1 Like

Great! I appreciate it. Thanks a lot!

This is all it needs to read:

And there are more very good reads about why systemd was embraced by every major distro.

EDIT: Typo in “embraced”

2 Likes

Embraced or embarrassed? :wink:

1 Like

This is why I am using dnsmasq. pihole is build upon dnsmasq. But I did not want to install a separate pihole instance so I just installed dnsmasq.

Sure, i’m aware of that whole argument line and don’t deny why it was embraced all over the place (i’d name that universal api), however i fully deny how it was implemented / embraced.

You can disable or uninstall sytemd-resolved when not using VPN e.g. NordVPN, then install the old openresolv or something else what you know.


Edit:

You can check:

sudo systemd-resolve --status

This article has some interesting wording.

e.g… “Instead of the systemd, it uses the real init system in order to keep the OS simple and user-friendly”

The “real init system” is funny. And implying that systemd is not simple and user-friendly is simply wrong. I find systemd a lot simpler and user friendly than anything I have used before. But ok. I guess we all get used to statements like this. And it is off topic here anyways.

3 Likes

You are right!
That’s on the writer of the article. Just posted it as a reference list to some non-systemd systems.
Not that I necessarily subscribe to the opinions of the author.

systemd-resolved works as advertised - been using it for a long time - no issues. Some VPN will not reset nameserver entries correct if another resolver is used.

The setup is extremely simple

sudo systemctl enable --now systemd-resolved

Move you existing resolv.conf out of the way

sudo mv /etc/resolv.conf /etc/resolv.conf.bak

Create a symlink to the stub

sudo ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

If you are running into issues with slow lookups it is easily caused by how a resolver works.

If the first configured nameserver doesn’t answer - the next is tried - if it answers it becomes the default - which may cause slow lookups depending on which nameserver is used as fallback. All that can be queried by

resolvectl status

I see you are familiar with the Arch documentation on systemd-resolved.

If you want to use other fallbacks it is as easy as a drop-in config in /etc/systemd/resolved.conf.d/ and define your resolvers.

I even changed DNS from “NextDNS” to “AdguardDNS”. Nope, still have that awful lag with systemd-resolved.

I have setup (with hints from a blog) knot-resolver to use cloudflare and block lists using the “rpz” policy.

❯ tree .
.
├── aria2.log
├── EnergizedAdult.rpz
├── EnergizedSpark.rpz
├── kresd.conf
└── root.hints

0 directories, 5 files

❯ cat kresd.conf 
-- SPDX-License-Identifier: CC0-1.0
-- vim:syntax=lua:set ts=4 sw=4:
-- Refer to manual: https://knot-resolver.readthedocs.org/en/stable/

-- Network interface configuration
net.listen('127.0.0.1', 53, { kind = 'dns' })
-- net.listen('127.0.0.1', 853, { kind = 'tls' })
-- net.listen('127.0.0.1', 443, { kind = 'doh2' })
-- net.listen('::1', 53, { kind = 'dns', freebind = true })
-- net.listen('::1', 853, { kind = 'tls', freebind = true })
-- net.listen('::1', 443, { kind = 'doh2' })

-- Load useful modules
modules = {
	'policy',
  'hints > iterate',  -- Allow loading /etc/hosts or custom root hints
  'stats',            -- Track internal statistics
	'predict'          -- Prefetch expiring/frequent records
}

-- Cache size
cache.size = 100 * MB

policy.add(
	policy.all(
	policy.TLS_FORWARD(
		{
			{'1.1.1.1', hostname='one.one.one.one'},
			{'9.9.9.9', hostname='dns.quad9.net'}
})))

policy.add(policy.rpz(policy.DENY, '/etc/knot-resolver/EnergizedSpark.rpz'))

policy.add(policy.rpz(policy.DENY, '/etc/knot-resolver/EnergizedAdult.rpz'))

knot-resolved is so snappy compared to systemd-resolved!!!

systemd can pry knot-resolved from my cold dead hands.

1 Like

:slight_smile:

I wouldn’t try to convice you otherwise - but your experience with systemd-networkd is hardly a justifiable cause to go on a religios rant against systemd.

If would have been better to find the real cause of your issues instead of ranting and blaming systemd.

Over the years my local network has grown and the host lookups frequently failed even though a used a local domain.

A couple of years back I setup a subnet of one of my internet domains.

Then I added a NS record at my hosting provider pointing back into my network
I set up a local DNS on a raspberry pi using bind and rpz to filter bad domains
I setup a isc-dhcp service on the pi and assign static ip’s based on mac address
The device as run on the same sdcard for 3y now

Issues with NordVPN not restoring my resolv.conf correct on disconnect made me switch my systems from openresolv to use systemd-resolved.

As is evident for anyone from this thread I did none of that; maybe I tried to be humorous about it but that’s that. I even said it’s very convenient. I use systemd and no other init system.

I always thought what’s baked in is the best solution; I kind of like the idea of it. But sometimes it’s not and I wanted to share it with the community and maybe find a solution.

Maybe all this is a miscommunication and I am sure we have the best intentions. :slightly_smiling_face:

Anyway I suspect the lag I experience is because I use DNS over LTS instead of plain DNS with systemd-resolved. What do you use in your setup? If you use plain DNS, and if at all possible, can you try this config and see if there is a lag at least when you open a web page for the first time?

[Resolve]
DNS=45.90.28.0#3faf98.dns1.nextdns.io
DNS=2a07:a8c0::#3faf98.dns1.nextdns.io
DNS=45.90.30.0#3faf98.dns2.nextdns.io
DNS=2a07:a8c1::#3faf98.dns2.nextdns.io
DNSOverTLS=yes

(https://my.nextdns.io/3faf98/setup)

tell me which domains to lookup - you will most likely know some I have never heard of :slight_smile:

I should mention I don’t use IPv6 - so I commented the IPv6 adresses - reason for not using IPv6 - it slows my network.

$ resolvectl status
Global
           Protocols: +LLMNR +mDNS +DNSOverTLS DNSSEC=no/unsupported
    resolv.conf mode: stub
  Current DNS Server: 45.90.28.0#3faf98.dns1.nextdns.io
         DNS Servers: 45.90.28.0#3faf98.dns1.nextdns.io 45.90.30.0#3faf98.dns2.nextdns.io
Fallback DNS Servers: 1.1.1.1#cloudflare-dns.com 9.9.9.9#dns.quad9.net 8.8.8.8#dns.google
                      2606:4700:4700::1111#cloudflare-dns.com 2620:fe::9#dns.quad9.net 2001:4860:4860::8888#dns.google

Link 2 (eno1)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
         Protocols: +DefaultRoute +LLMNR -mDNS +DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 172.30.xx.yy
       DNS Servers: 172.30.xx.yy
        DNS Domain: net.nix.dk

Looking at the DNS servers you provided there is something that seems off - the address appears very much off - 45.90.28.0 does not correspond to hostname

45.90.28.0#3faf98.dns1.nextdns.io

This is likely to cause slow lookups until the resolver finds a responding nameserver

$ drill dns1.nextdns.io
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 59589
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; dns1.nextdns.io.     IN      A

;; ANSWER SECTION:
dns1.nextdns.io.        41      IN      CNAME   dns1.steering.nextdns.io.
dns1.steering.nextdns.io.       60      IN      A       188.172.192.71

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 89 msec
;; SERVER: 127.0.0.53
;; WHEN: Sat May 28 12:48:48 2022
;; MSG SIZE  rcvd: 77
$ drill 3faf98.dns1.nextdns.io
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 21814
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; 3faf98.dns1.nextdns.io.      IN      A

;; ANSWER SECTION:
3faf98.dns1.nextdns.io. 30      IN      CNAME   dns1.steering.nextdns.io.
dns1.steering.nextdns.io.       60      IN      A       188.172.192.71

Doing a bit of testing the address appear to be an anycast address.


Continued testing reveals no delays using systemd-resolved with the config you provided.

I got lookups around 50-150ms

When I use your config I get

$ drill manjaro.br
Error: error sending query: Could not send or receive, because of network error

If I time that

$ time drill manjaro.br
Error: error sending query: Could not send or receive, because of network error

real    0m15,016s
user    0m0,003s
sys     0m0,000s

Wherea when I use my default setup I get

$ drill manjaro.br
;; ->>HEADER<<- opcode: QUERY, rcode: NXDOMAIN, id: 1384
;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 
;; QUESTION SECTION:
;; manjaro.br.  IN      A

;; ANSWER SECTION:

;; AUTHORITY SECTION:
br.     10800   IN      SOA     a.dns.br. hostmaster.registro.br. 2022148244 1800 900 604800 900

;; ADDITIONAL SECTION:

;; Query time: 218 msec
;; SERVER: 127.0.0.53
;; WHEN: Sat May 28 13:25:35 2022
;; MSG SIZE  rcvd: 90

I guess on my standard install it gives me this.

[ricklinux@rick-ms7c37 ~]$ resolvectl status
Failed to get global data: Unit dbus-org.freedesktop.resolve1.service not found.

All of those commands come up instantly. :thinking:

the service needs to be enabled for resolvectl to work

systemctl status systemd-resolved

What is the reason i would even want to use it?