Trouble with dual Nvidia GPUs

To get the formalities out of the way, yes, I did attempt to RTFM, Google/Bing/AskJeeves, f#ck around and find out, et cetera.

I have been enjoying my time using EndeavourOS for the past year across several laptops. My New Year’s resolution was to finally commit to installing Linux to my main rig, in an attempt to replace my daily driver Windows install. However the biggest hurdle has been the fact that I run two Nvidia GPUs, a GTX 960 for basic tasks, and an RTX 2070 for the more demanding loads. No matter what I seem to do, I cannot get both GPUs to meaningfully output a display to the monitors connected to them simultaneously. I tried installing the official Nvidia Drivers, fiddling with Nvidia-Settings, but the best I can do is get one X server functional, and a second one that hangs after Plasma loads, and has an “X” cursor on it.
I waited this long to make an account on these forums because I often detest making throwaway accounts for one-off problems, but I’ve been banging my head against this wall for months, and now all-of-a-sudden, which GPU/X Server works and which does not has swapped.
Any and all help is appreciated.

Well…I used to run a custom xorg.conf for my SLI 1070’s…I would think the only way you could make it work is to define which monitors use which video cards…

Take a look at: https://superuser.com/questions/117239/how-can-i-get-multiple-video-cards-to-work-on-linux
https://wiki.archlinux.org/title/Multihead
https://bbs.archlinux.org/viewtopic.php?id=235565
https://wiki.gentoo.org/wiki/Xorg/Multiple_monitors
https://www.x.org/releases/current/doc/man/man5/xorg.conf.5.xhtml
https://wiki.gentoo.org/wiki/Xorg.conf
https://www.linux.com/news/editing-basics-xorgconf-file/

Some of these are old…you need to use the “modern” /etc/X11/xorg.conf.d location & do each section as a separate xxxx.conf file. It will take a bit of trial & error to get what you want…but I’ve made stuff that worked & still run .conf mods there.

My current /etc/X11/xorg.conf.d has 3 custom conf files…1 for the mods to my keyboard 00-keyboard.conf custom nvidia settings 10-nvidia,conf and custom mouse settings 50-vmmouse.conf

The nvidia conf looks like this:

#
Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "DP-0"
    ModelName      "GBT G34WQC"
    HorizSync       222.0 - 222.0
    VertRefresh     48.0 - 144.0
    Option         "DPMS"
EndSection

Section "Monitor"
    Identifier     "HDMI-0"
    ModelName      "Acer P201W"
    VertRefresh     60.0 - 75.0
    Option         "RightOf"   "DP-1"
    Option         "DPMS"
EndSection

Section "OutputClass"
    Identifier    "nvidia"
    MatchDriver   "nvidia-drm"
    Driver        "nvidia"
    Option        "Coolbits" "16"
Endsection

Section "Device"
    Identifier     "Device0"
    Option         "Monitor-DFP-1" "DP-0"
    Option         "Monitor-DFP-0" "HDMI-0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "NVIDIA GeForce RTX 3070"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "DFP-1"
    Option         "metamodes" "DP-0: 3440x1440_144 +0+0, HDMI-0: nvidia-auto-select +3440+169"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

As you can see, I wanted both of the monitors to be in a certain order since the ports on my card would put the wrong monitor first…I wanted DP before HDMI & I wanted the smaller monitor to be centered to the right of the larger monitor.

a progress update of sorts:
Previously, the GTX 960 ran flawlessly with the RTX 2070 giving me crap with the X Screen/Nvidia-settings tomfoolery. I then did a PC Case swap, and now all of a sudden the two swapped. They are both connected to the same PCIe slots they were before, but I did have to play musical chairs to get them to both show up in my system after some other upgrades were made (extra peripherals). Ironically, the information I have learned in the past 36 hours would have completely solved my issues with the RTX 2070 being stubborn.

What I have learned so far is that I can run DISPLAY=0.# [program] to force run a program on the other video card (where # is the number of the monitor). However, to get it to barely work in a mostly usable manner, I needed to set each monitor as its own X Screen. I say “barely usable” because no matter what I have done so far, I cannot get any programs ran this way to unscale from my RTX 2070’s 4K DPI, so most UI is really big. Additionally, I do not get the system’s context menu. Program specific ones work.
I attempted in vain to try running plasma or kwin through the command above, but that did not work.

My only remaining ideas are to:
A- play musical chairs again until I somehow get the GTX 960 back in control
or
B- take the nuclear approach and hope a clean install fixes everything (not guaranteed, and I will need to go through the headache of manually installing LightScribe software again)

But for now I’m going to take a break before I lose too many braincells.

Did you install this from the AUR?

Tried to. the LaCie stuff was removed from the AUR due to the source URL being down, but by using the Wayback Machine, the .rpm was still able to be downloaded, so I had to learn how to substitute in a local file in the PKGBUILD instead of pinging a dead URL.

This one?

https://aur.archlinux.org/packages/lightscribe

No, this one.
https://aur.archlinux.org/packages/4l

https://aur.archlinux.org/packages/lightscribe-labeler

I remember trying to install that one too. All I know is I got 4L installed eventually.

What is your current output for the gpu?

inxi -Ga
Graphics:
  Device-1: NVIDIA TU106 [GeForce RTX 2070] vendor: ASUSTeK
    driver: nvidia v: 535.86.05 alternate: nouveau,nvidia_drm
    non-free: 535.xx+ status: current (as of 2023-07) arch: Turing
    code: TUxxx process: TSMC 12nm FF built: 2018-22 pcie: gen: 1
    speed: 2.5 GT/s lanes: 8 link-max: gen: 3 speed: 8 GT/s
    lanes: 16 bus-ID: 07:00.0 chip-ID: 10de:1f02 class-ID: 0300
  Device-2: NVIDIA GM206 [GeForce GTX 960] vendor: Micro-Star MSI
    driver: nvidia v: 535.86.05 alternate: nouveau,nvidia_drm
    non-free: 535.xx+ status: current (as of 2023-07) arch: Maxwell
    code: GMxxx process: TSMC 28nm built: 2014-19 pcie: gen: 3
    speed: 8 GT/s lanes: 4 link-max: lanes: 16 bus-ID: 08:00.0
    chip-ID: 10de:1401 class-ID: 0300
  Device-3: TANDBERG PrecisionHD Camera
    driver: snd-usb-audio,uvcvideo type: USB rev: 2.0 speed: 480 Mb/s
    lanes: 1 mode: 2.0 bus-ID: 5-2.4:5 chip-ID: 1f82:0001
    class-ID: 0102 serial: AA1AP14F00510
  Device-4: EVGA XR1 Lite Capture Box Video
    driver: hid-generic,snd-usb-audio,usbhid,uvcvideo type: USB
    rev: 3.0 speed: 5 Gb/s lanes: 1 mode: 3.2 gen-1x1 bus-ID: 6-1:5
    chip-ID: 3842:310a class-ID: 0300 serial: 385203910510
  Display: x11 server: X.Org v: 21.1.8 compositor: kwin_x11
    driver: X: loaded: nvidia gpu: nvidia,nvidia display-ID: :0
    screens: 4 default screen: 0
  Screen-1: 0 s-res: 3840x2160 s-dpi: 157
    s-size: 621x341mm (24.45x13.43") s-diag: 708mm (27.89")
  Monitor-1: DP-2 res: 3840x2160 hz: 60 dpi: 157
    size: 621x341mm (24.45x13.43") diag: 708mm (27.89") modes: N/A
  Screen-2: 1 s-res: 1080x1920 s-dpi: 95
    s-size: 290x518mm (11.42x20.39") s-diag: 594mm (23.37")
  Screen-3: 2 s-res: 1920x1080 s-dpi: 101
    s-size: 483x272mm (19.02x10.71") s-diag: 554mm (21.82")
  Screen-4: 3 s-res: 1024x1280 s-dpi: 86
    s-size: 301x382mm (11.85x15.04") s-diag: 486mm (19.15")
  API: OpenGL v: 4.6.0 NVIDIA 535.86.05 renderer: NVIDIA GeForce
    RTX 2070/PCIe/SSE2 direct-render: Yes

Not sure why it included my webcam and capture card.

So nvidia drivers are installed and it’s rendering on the RTX 2070. So all you need to do is get it to output properly to the monitors?

Graphics:
  Device-1: NVIDIA TU106 [GeForce RTX 2070] vendor: ASUSTeK
    driver: nvidia v: 535.86.05 alternate: nouveau,nvidia_drm
    non-free: 535.xx+ status: current (as of 2023-07) arch: Turing
    code: TUxxx process: TSMC 12nm FF built: 2018-22 pcie: gen: 1
    speed: 2.5 GT/s lanes: 8 link-max: gen: 3 speed: 8 GT/s
    lanes: 16 bus-ID: 07:00.0 chip-ID: 10de:1f02 class-ID: 0300
 
 Device-2: NVIDIA GM206 [GeForce GTX 960] vendor: Micro-Star MSI
    driver: nvidia v: 535.86.05 alternate: nouveau,nvidia_drm
    non-free: 535.xx+ status: current (as of 2023-07) arch: Maxwell
    code: GMxxx process: TSMC 28nm built: 2014-19 pcie: gen: 3
    speed: 8 GT/s lanes: 4 link-max: lanes: 16 bus-ID: 08:00.0
    chip-ID: 10de:1401 class-ID: 0300
API: OpenGL v: 4.6.0 NVIDIA 535.86.05 renderer: NVIDIA GeForce
    RTX 2070/PCIe/SSE2 direct-render: Yes

Ideally I would prefer to render most of my system on the GTX 960 (the 3 monitors connected to it), and save my RTX 2070 for more demanding tasks like 4K Gaming/Virtualization/Video Rendering/Compute.

Update - I’ve tried a few things.

I’ve tried reseating the cards in every order possible. I’ve tried halving the bandwidth to see if that de-prioritizes the 2070. I’ve tried reinstalling my old M.2 that I transferred my install from during the case swap (capacity upgrade). The only ounce of success I’ve had was removing the 2070 from the xorg.conf

Somehow, someway, that got one of my monitors on my 960 to work. I will attach said xorg.conf tomorrow after work. I would now, however I am writing this from my phone as I ready myself for work tomorrow.

What is your specific hardware? (motherboard)

inxi -Faz | eos-sendlog

I’m going to give you a link here to the Nvidia Installation Guide. Not sure if your hardware does SLI for dual graphics and or maybe you can offload graphics rendering with randr like a Hybrid system can to trick it into using a specific gpu. Not sure how you would specify certain programs to use the 2070 unless you could use prime? or? :man_shrugging:

Anyway i hope this helps.

https://download.nvidia.com/XFree86/Linux-x86_64/396.51/README/index.html

Here is the output. If it’s missing any crucial info due to me not appending doas, I can do that if needed.
http://ix.io/4Cil

Additionally, here is the contents of my xorg.conf

# nvidia-settings: X configuration file generated by nvidia-settings
# nvidia-settings:  version 535.86.05


Section "ServerLayout"

# Removed Option "Xinerama" "0"
    Identifier     "Layout0"
    Screen      0  "Screen0" 4024 0
    Screen      1  "Screen1" 2944 120
    Screen      2  "Screen2" 1024 540
    Screen      3  "Screen3" 0 440
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "0"
EndSection

Section "Files"
EndSection

Section "InputDevice"

    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"

    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"

    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "BenQ EL2870U"
    HorizSync       135.0 - 135.0
    VertRefresh     40.0 - 60.0
    Option         "DPMS"
EndSection

Section "Monitor"

    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor1"
    VendorName     "Unknown"
    ModelName      "DELL U2312HM"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 76.0
    Option         "DPMS"
EndSection

Section "Monitor"
    Identifier     "Monitor2"
    VendorName     "Unknown"
    ModelName      "DELL P2211H"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 76.0
    Option         "DPMS"
EndSection

Section "Monitor"
    Identifier     "Monitor3"
    VendorName     "Unknown"
    ModelName      "HP LA1951"
    HorizSync       24.0 - 83.0
    VertRefresh     50.0 - 76.0
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "NVIDIA GeForce RTX 2070"
    BusID          "PCI:7:0:0"
EndSection

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "NVIDIA GeForce GTX 960"
    BusID          "PCI:8:0:0"
    Screen          0
EndSection

Section "Device"
    Identifier     "Device2"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "NVIDIA GeForce GTX 960"
    BusID          "PCI:8:0:0"
    Screen          1
EndSection

Section "Device"
    Identifier     "Device3"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "NVIDIA GeForce GTX 960"
    BusID          "PCI:8:0:0"
    Screen          2
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "DFP-3"
    Option         "metamodes" "nvidia-auto-select +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"

# Removed Option "metamodes" "DVI-I-1: nvidia-auto-select +0+0"
# Removed Option "metamodes" "DVI-I-1: nvidia-auto-select +0+0 {rotation=left}"
# Removed Option "metamodes" "DVI-I-1: nvidia-auto-select +2944+0 {rotation=left}, HDMI-0: nvidia-auto-select +1024+420, DP-1: nvidia-auto-select +0+320 {rotation=left}"
    Identifier     "Screen1"
    Device         "Device1"
    Monitor        "Monitor1"
    DefaultDepth    24
    Option         "nvidiaXineramaInfoOrder" "DFP-1"
    Option         "Stereo" "0"
    Option         "metamodes" "DVI-I-1: nvidia-auto-select +0+0 {rotation=left}"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"

# Removed Option "metamodes" "HDMI-0: nvidia-auto-select +0+0"
    Identifier     "Screen2"
    Device         "Device2"
    Monitor        "Monitor2"
    DefaultDepth    24
    Option         "nvidiaXineramaInfoOrder" "DFP-1"
    Option         "Stereo" "0"
    Option         "metamodes" "HDMI-0: nvidia-auto-select +0+0 {AllowGSYNC=Off}"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"

# Removed Option "metamodes" "DP-1: nvidia-auto-select +0+0"
    Identifier     "Screen3"
    Device         "Device3"
    Monitor        "Monitor3"
    DefaultDepth    24
    Option         "nvidiaXineramaInfoOrder" "DFP-3"
    Option         "Stereo" "0"
    Option         "metamodes" "DP-1: nvidia-auto-select +0+0 {rotation=left, AllowGSYNC=Off}"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection


The next thing I was going to try was to rename it, and let nvidia-settings create a clean one that I can work with, since the one above has remnants of my previous endeavors… hehe, endeavors.

Normally there shouldn’t be an xorg file unless you create one for the screen layout. There should only be this file.

/etc/X11/xorg.conf.d/20-nvidia.conf

https://wiki.archlinux.org/title/NVIDIA#Xorg_configuration

Edit: I’m no expert at this either just trying to help.

Well, I don’t know what else to say except that there is no 20-nvidia.conf in /etc/X11/xorg.conf.d/.Perhaps there was one prior to me first using nvidia-settings, but all that is in that directory is 00-keyboard.conf.

It could be because that is created when you install the nvidia drivers. If you are using xsettings in nvidia it will take over i think or overwrite it? Not sure? :man_shrugging: