Performance Issues with NVIDIA-GPU on Fresh Install

That sounds like you removed nvidia-dkms (or otherwise don’t have NVIDIA driver installed for 5.4).

Did you install nvidia and then didn’t install nvidia-lts for the LTS kernel?