huh…so…it acts like its working, my switch isn’t configured yet, long story, fighting trying to update the software version, so I plugged both pcs to each other and … so how the heck do I add it as a device in the EOS network manager if its just gonna act like its working.
[ 11.202972] mlx5_core 0000:52:00.0: firmware version: 12.28.2302
[ 11.203001] mlx5_core 0000:52:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 0000:50:04.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
[ 13.488040] mlx5_core 0000:52:00.0: Port module event: module 0, Cable unplugged
[volt@Editing-PC ~]$ sudo dmesg | grep mlx
[sudo] password for volt:
[ 11.202972] mlx5_core 0000:52:00.0: firmware version: 12.28.2302
[ 11.203001] mlx5_core 0000:52:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 0000:50:04.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
[ 13.488040] mlx5_core 0000:52:00.0: Port module event: module 0, Cable unplugged
[10268.855997] mlx5_core 0000:52:00.0: Port module event: module 0, Cable plugged
[volt@Editing-PC ~]$
bonus points to this showing me I need to swap what port the things plugged into. good to discover that so thank you for that bonus discovery.
[volt@Editing-PC ~]$ mst
-E- You must be root to use mst tool
[volt@Editing-PC ~]$ sudo mst
[sudo] password for volt:
Usage:
/usr/bin/mst {start|stop|status|remote|server|restart|save|load|rm|add|help|version|gearbox|cable}
Type "/usr/bin/mst help" for detailed help
[volt@Editing-PC ~]$ sudo mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
Unloading MST PCI module (unused) - Success
[volt@Editing-PC ~]$
I have, though I must admit, I do not know what I’m supposed to do with it, that was one of the commands that worked while others did not, again not entirely sure which ones didn’t but I do recall trying to follow guides suggesting to run XYZ commands and some of the latter ones for configuration did not work while the prior ones did.
Your card might be in Infiniband mode instead Ethernet. Try sudo mst status maybe? I don’t have an Arch box available nor do I have Mellanox gear so I’m just going with basic Linux knowledge, a bit of internet and a lot of guesswork xD
I know for sure it is in ethernet mode, that was one of the few commands that worked, and I’ve set it that way in bios on both pcs, though that command did possibly lend some insight to another issue plauging me it seems. though again I’m unsure how to fix it, and guess work or not, you’ve been far more help than I’m being with my own knowledge, so still, thank you.
[volt@Editing-PC ~]$ sudo mst status
MST modules:
------------
MST PCI module is not loaded
MST PCI configuration module loaded
MST devices:
------------
/dev/mst/mt4115_pciconf0 - PCI configuration cycles access.
domain:bus:dev.fn=0000:52:00.0 addr.reg=88 data.reg=92 cr_bar.gw_offset=-1
Chip revision is: 00
[volt@Editing-PC ~]$
admittedly I’m far more savvy with ubuntu/debian than EOS but I’m liking EOS far more than either option and it was a friend that suggested to me to use EOS and I’m glad he did, minus some windows to linux learning curves its been nothing but more enjoyable than using microsoft and all the BS that entails.
I tried it twice, thinking maybe I typod, first using my own, the second copying yours, both the same thing.
[volt@Editing-PC ~]$ sudo mlxconfig -d /dev/mt4115_pciconf0 q
-E- Failed to open the device
[volt@Editing-PC ~]$ sudo mlxconfig -d /dev/mt4115_pciconf0 q
-E- Failed to open the device
[volt@Editing-PC ~]$
Or maybe you have to run sudo mst start in one terminal, keep it running, and then try mlxconfig in another window? I’m sorry, I’m running out of knowledge. If nothing helps, reboot and see if that fixed somehting
… well that just got interesting. not sure if it worked or not yet, but apparently using nmcli to issue a command on a test basis, it added a custom named “interface” in the network manager using the device name found earlier, so…maybe that was the only thing missing and I’m just overcomplicating things??? I’m gonna see if I can replicate this on the other pc and if it all works and I can talk between them using a pc to pc loopback cable…then cool, I’ll be stupid happy and report accordingly, time to try and reproduce this first since I don’t have the switch to use to test it with. thank you!
That being the test I just tried that supposedly got it to show up in the network manager…unsure yet if that’s actually what I think it is or it just let me make a bad entry
crap. no that did not seem to do what I thought since hitting the dropdown under “restrict to device” still only shows the existing entries and not the new one I’m trying to add, that being the 100 gig card.
I don’t know the answer but I sure as hell hope its gonna work.
Also, out of curiosity: I saw your device name is Editing-PC
Did you buy datacenter networking gear just to be able to move around videos faster? What humongous videos are you working with?
I mean, multiple different things, but faster file transfers yes, I have a storage server and the spinning rust in the thing is fast enough to fully saturate a 10 gig fiber link so I wanted to get something faster since using nothing to speed it up, just the raw rust transfer speeds, was enough to fully saturate it both ways, and I’m building a bigger faster one with more caching and stupid amounts of ram? if 10 gig is already the bottleneck just for raw rust transfers… I need/want faster since I’m already capable of exceeding that much. I’m just very impatient, and also do a lot of audio-video editing for a podcast I’ve helped run for 10+ years, my motivation to work on file related things disappears fast, so if I can make it less of a problem by speeding the file process up, I’m going to do so.
I accidentally made a monster and I cant be happier. 17 “disk” raid 10 in truenas, so, 17 mirrors…its not a shock after learning more WHY It ends up being such a monster, happy mistake, I cant take credit for that, I had no idea what I was doing and it ended in the best possible way.
okay so trying to replicate things on the other pc so far, both of them have something in common, though one of the two pci modules are there on both, both systems for whatever reason say “module mst_pci not found” when running sudo mst start
okay wait I lied, running sudo mst status on the other one shows neither module is loaded.
[volt@GamingPC ~]$ sudo mst
Usage:
/usr/bin/mst {start|stop|status|remote|server|restart|save|load|rm|add|help|version|gearbox|cable}
Type "/usr/bin/mst help" for detailed help
[volt@GamingPC ~]$ sudo mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI modulemodprobe: FATAL: Module mst_pci not found in directory /lib/modules/6.16.8-arch3-1
- Failure: 1
Loading MST PCI configuration modulemodprobe: FATAL: Module mst_pciconf not found in directory /lib/modules/6.16.8-arch3-1
- Failure: 1
Create devices
mst_pci driver not found
Unloading MST PCI module (unused)modprobe: FATAL: Module mst_pci not found.
- Failure: 1
Unloading MST PCI configuration module (unused)modprobe: FATAL: Module mst_pciconf not found.
- Failure: 1
[volt@GamingPC ~]$ sudo mst
Usage:
/usr/bin/mst {start|stop|status|remote|server|restart|save|load|rm|add|help|version|gearbox|cable}
Type "/usr/bin/mst help" for detailed help
[volt@GamingPC ~]$ sudo mst status
MST modules:
------------
MST PCI module is not loaded
MST PCI configuration module is not loaded
PCI Devices:
------------
03:00.0
[volt@GamingPC ~]$
[volt@Editing-PC ~]$ sudo mst status
[sudo] password for volt:
MST modules:
------------
MST PCI module is not loaded
MST PCI configuration module loaded
MST devices:
------------
/dev/mst/mt4115_pciconf0 - PCI configuration cycles access.
domain:bus:dev.fn=0000:52:00.0 addr.reg=88 data.reg=92 cr_bar.gw_offset=-1
Chip revision is: 00
[volt@Editing-PC ~]$
As long as you don’t need to dive deep into configuring your cards thats fine. Afaik thats a kernel module needed to configure Mellanox cards. If it doesn’t become an issue I wouldn’t worry about it
Could you redo 
