Free RAM is wasted RAM.
Since 
, or any distro in general has almost nothing to do when there’s no DE installed (idle), it uses all that free RAM as cache. Obviously, it leaves some free, otherwise if you tried to load something it’d crash.
On your laptop, there’s a lot less free RAM (due to a graphical session, which could be games/videos/development etc.), so not as much ends up as buff/cache and ends up in used. Both machines are using free RAM as cache (which is great), but the server is doing it more than the laptop, since the laptop is using more memory for non-cache things.