How to get NVidia Pascal running fast? (I want Nouveau not llvmpipe)

i do think the the main issue is the nouveau drivber for you related to your GPU Model.
I have a 3060 here and if i run this one on nouveau and vulkan it is running at big speeds fully usable even in Games.

The way to go for yor issue is (on Arch/based) currently to use the legacy driver from AUR nvidia-580xx-dkms nvidia-580xx-utils and indeed remove vulkan-nouveau xf86-video-nouveau

what you may hit is something like this too:

but if i were you i would first switch to 580 legacy drivers and see what happen.