Hello everyone,
I have to use EndeavourOS via RDP screen-sharing (I do that via GNOME 42 sharing) for large periods of the day, and connect to it from a Windows client. One of the things that annoys me, is that this does not forward the audio from EndeavourOS to Windows.
I ducked it out on the web and found that you can use ffmpeg to send audio and ffplay to receive it. I’ve managed to get it working with MP3 codec but that has extreme latency. Since I can play video on that machine and receive it in real-time, I expect this is because of buffering, so I tried to switch to a different codec, but nothing works.
I will describe what worked for me and where I am stuck. Note that I relied on this gist and this guide
So to attach to a device I listed my devices:
$ pactl list short sources
1842 alsa_output.pci-0000_01_00.1.hdmi-stereo-extra4.monitor PipeWire s32le 2ch 48000Hz RUNNING
$ export AUDIO_DEV="alsa_output.pci-0000_01_00.1.hdmi-stereo-extra4.monitor"
So, using this environment variable, I ended up with the below combination of commands (same as the gist) which works for me when using MP3. I can get audio from EndeavourOS to play back on my Windows client, but the latency is 3 seconds:
# on Linux side (capture and send audio)
$ ffmpeg -re -f pulse -i "$AUDIO_DEV" -acodec mp3 -f rtp rtp://192.168.9.166:1234 -sdp_file stream.sdp
# on Windows side (receive and play back locally)
> ffplay "rtp://192.168.9.166:1234"
The latency is always about 3 seconds. It doesn’t matter if I start playing a Youtube video or simply running the sound test (“front left / front right” from sound settings) it’s the same. Since the video responsiveness is near-instant, I think there’s some buffering happening by ffmpeg, so I followed the advice in the gist to use PCM encoding instead:
# on Linux side (capture and send audio)
$ ffmpeg -re -f pulse -i "$AUDIO_DEV" -ac 2 -acodec pcm_s16le -ar 48000 -f s16le "udp://192.168.9.166:1234"
# on Windows side (receive and play back locally)
> ffplay -ac 2 -acodec pcm_s16le -ar 48000 -f s16le -analyzeduration 0 -probesize 32 -i "udp://192.168.9.166:1234?listen=1"
This also works and I still get the audio, but again the latency is again quite high (still about 2-3 seconds).
I found some extra flags in this SO post for reducing latency (-fflags nobuffer -flags low_delay -framedrop) but they did not have any effect.
Since the network speed/latency is very low between the source and destination (same network) I think the problem is likely on the capture/encoding part of the Linux sender? The only thing I changed from the gist is the fact that I use the physical audio device as input rather than a virtual one, so I tried adding a virtual device as well:
$ pactl load-module module-null-sink sink_name=remote
536870913
$ pactl list short sources
1842 alsa_output.pci-0000_01_00.1.hdmi-stereo-extra4.monitor PipeWire s32le 2ch 48000Hz SUSPENDED
2149 remote.monitor PipeWire float32le 2ch 44100Hz SUSPENDED
$ ffmpeg -re -f pulse -i "remote.monitor" -ac 2 -acodec pcm_s16le -ar 48000 -f s16le "udp://192.168.9.166:1234"
This also worked (I had to switch my default audio device to “remote”) but the latency is no better.
Is there any way I can improve the delay between sender and receiver? Again, the network conditions are perfect, ping time is very low (2ms), and when I press on a video in Youtube, the video starts playing back immediately and I can see it on the client, but the audio arrives 2-3 seconds later.
At this stage I’m not sure where the delay is (is the server buffering before transmitting? is the client buffering before playback?) and I’ve no idea how to figure out / debug this…