When I ran DaVinci in the terminal, nothing else printed after the Accepted new client line. That means that if i cause the error, no info about the error is printed to the terminal.
There is a nvidia-smi -q output:
==============NVSMI LOG==============
Timestamp : Fri Apr 5 21:42:55 2024
Driver Version : 550.67
CUDA Version : 12.4
Attached GPUs : 1
GPU 00000000:01:00.0
Product Name : NVIDIA GeForce RTX 3050 Laptop GPU
Product Brand : GeForce
Product Architecture : Ampere
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : Disabled
Addressing Mode : None
MIG Mode
Current : N/A
Pending : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : N/A
Pending : N/A
Serial Number : N/A
GPU UUID : GPU-2bc4f72e-17b1-d1fd-447f-38e7d0391e10
Minor Number : 0
VBIOS Version : 94.07.3A.00.B2
MultiGPU Board : No
Board ID : 0x100
Board Part Number : N/A
GPU Part Number : 25A2-725-A1
FRU Part Number : N/A
Module ID : 1
Inforom Version
Image Version : G001.0000.03.03
OEM Object : 2.0
ECC Object : N/A
Power Management Object : N/A
Inforom BBX Object Flush
Latest Timestamp : N/A
Latest Duration : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU C2C Mode : N/A
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
vGPU Heterogeneous Mode : N/A
GPU Reset Status
Reset Required : No
Drain and Reset Recommended : N/A
GSP Firmware Version : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x01
Device : 0x00
Domain : 0x0000
Base Classcode : 0x3
Sub Classcode : 0x0
Device Id : 0x25A210DE
Bus Id : 00000000:01:00.0
Sub System Id : 0x88F9103C
GPU Link Info
PCIe Generation
Max : 4
Current : 1
Device Current : 1
Device Max : 4
Host Max : 4
Link Width
Max : 16x
Current : 8x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 0 KB/s
Rx Throughput : 0 KB/s
Atomic Caps Inbound : N/A
Atomic Caps Outbound : N/A
Fan Speed : N/A
Performance State : P8
Clocks Event Reasons
Idle : Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
Sparse Operation Mode : N/A
FB Memory Usage
Total : 4096 MiB
Reserved : 202 MiB
Used : 216 MiB
Free : 3680 MiB
BAR1 Memory Usage
Total : 4096 MiB
Used : 5 MiB
Free : 4091 MiB
Conf Compute Protected Memory Usage
Total : 0 MiB
Used : 0 MiB
Free : 0 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Encoder : 0 %
Decoder : 0 %
JPEG : 0 %
OFA : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
ECC Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
SRAM Correctable : N/A
SRAM Uncorrectable Parity : N/A
SRAM Uncorrectable SEC-DED : N/A
DRAM Correctable : N/A
DRAM Uncorrectable : N/A
Aggregate
SRAM Correctable : N/A
SRAM Uncorrectable Parity : N/A
SRAM Uncorrectable SEC-DED : N/A
DRAM Correctable : N/A
DRAM Uncorrectable : N/A
SRAM Threshold Exceeded : N/A
Aggregate Uncorrectable SRAM Sources
SRAM L2 : N/A
SRAM SM : N/A
SRAM Microcontroller : N/A
SRAM PCIE : N/A
SRAM Other : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Remapped Rows : N/A
Temperature
GPU Current Temp : 49 C
GPU T.Limit Temp : N/A
GPU Shutdown Temp : 100 C
GPU Slowdown Temp : 97 C
GPU Max Operating Temp : 87 C
GPU Target Temperature : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
GPU Power Readings
Power Draw : 6.84 W
Current Power Limit : 60.00 W
Requested Power Limit : 60.00 W
Default Power Limit : 60.00 W
Min Power Limit : 1.00 W
Max Power Limit : 75.00 W
GPU Memory Power Readings
Power Draw : N/A
Module Power Readings
Power Draw : N/A
Current Power Limit : N/A
Requested Power Limit : N/A
Default Power Limit : N/A
Min Power Limit : N/A
Max Power Limit : N/A
Clocks
Graphics : 210 MHz
SM : 210 MHz
Memory : 405 MHz
Video : 555 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Deferred Clocks
Memory : N/A
Max Clocks
Graphics : 2100 MHz
SM : 2100 MHz
Memory : 6001 MHz
Video : 1950 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Voltage
Graphics : 650.000 mV
Fabric
State : N/A
Status : N/A
CliqueId : N/A
ClusterUUID : N/A
Health
Bandwidth : N/A
Processes
GPU instance ID : N/A
Compute instance ID : N/A
Process ID : 810
Type : G
Name : /usr/lib/Xorg
Used GPU Memory : 4 MiB
GPU instance ID : N/A
Compute instance ID : N/A
Process ID : 32188
Type : C
Name : ./resolve
Used GPU Memory : 202 MiB
There is a nvidia-smi -h output:
NVIDIA System Management Interface -- v550.67
NVSMI provides monitoring information for Tesla and select Quadro devices.
The data is presented in either a plain text or an XML format, via stdout or a file.
NVSMI also provides several management operations for changing the device state.
Note that the functionality of NVSMI is exposed through the NVML C-based
library. See the NVIDIA developer website for more information about NVML.
Python wrappers to NVML are also available. The output of NVSMI is
not guaranteed to be backwards compatible; NVML and the bindings are backwards
compatible.
http://developer.nvidia.com/nvidia-management-library-nvml/
http://pypi.python.org/pypi/nvidia-ml-py/
Supported products:
- Full Support
- All Tesla products, starting with the Kepler architecture
- All Quadro products, starting with the Kepler architecture
- All GRID products, starting with the Kepler architecture
- GeForce Titan products, starting with the Kepler architecture
- Limited Support
- All Geforce products, starting with the Kepler architecture
nvidia-smi [OPTION1 [ARG1]] [OPTION2 [ARG2]] ...
-h, --help Print usage information and exit.
--version Print version information and exit.
LIST OPTIONS:
-L, --list-gpus Display a list of GPUs connected to the system.
-B, --list-excluded-gpus Display a list of excluded GPUs in the system.
SUMMARY OPTIONS:
<no arguments> Show a summary of GPUs connected to the system.
[plus any of]
-i, --id= Target a specific GPU.
-f, --filename= Log to a specified file, rather than to stdout.
-l, --loop= Probe until Ctrl+C at specified second interval.
QUERY OPTIONS:
-q, --query Display GPU or Unit info.
[plus any of]
-u, --unit Show unit, rather than GPU, attributes.
-i, --id= Target a specific GPU or Unit.
-f, --filename= Log to a specified file, rather than to stdout.
-x, --xml-format Produce XML output.
--dtd When showing xml output, embed DTD.
-d, --display= Display only selected information: MEMORY,
UTILIZATION, ECC, TEMPERATURE, POWER, CLOCK,
COMPUTE, PIDS, PERFORMANCE, SUPPORTED_CLOCKS,
PAGE_RETIREMENT, ACCOUNTING, ENCODER_STATS,
SUPPORTED_GPU_TARGET_TEMP, VOLTAGE, FBC_STATS
ROW_REMAPPER, RESET_STATUS, GSP_FIRMWARE_VERSION
Flags can be combined with comma e.g. ECC,POWER.
Sampling data with max/min/avg is also returned
for POWER, UTILIZATION and CLOCK display types.
Doesn't work with -u or -x flags.
-l, --loop= Probe until Ctrl+C at specified second interval.
-lms, --loop-ms= Probe until Ctrl+C at specified millisecond interval.
SELECTIVE QUERY OPTIONS:
Allows the caller to pass an explicit list of properties to query.
[one of]
--query-gpu Information about GPU.
Call --help-query-gpu for more info.
--query-supported-clocks List of supported clocks.
Call --help-query-supported-clocks for more info.
--query-compute-apps List of currently active compute processes.
Call --help-query-compute-apps for more info.
--query-accounted-apps List of accounted compute processes.
Call --help-query-accounted-apps for more info.
This query is not supported on vGPU host.
--query-retired-pages List of device memory pages that have been retired.
Call --help-query-retired-pages for more info.
--query-remapped-rows Information about remapped rows.
Call --help-query-remapped-rows for more info.
[mandatory]
--format= Comma separated list of format options:
csv - comma separated values (MANDATORY)
noheader - skip the first line with column headers
nounits - don't print units for numerical
values
[plus any of]
-i, --id= Target a specific GPU or Unit.
-f, --filename= Log to a specified file, rather than to stdout.
-l, --loop= Probe until Ctrl+C at specified second interval.
-lms, --loop-ms= Probe until Ctrl+C at specified millisecond interval.
DEVICE MODIFICATION OPTIONS:
[any one of]
-pm, --persistence-mode= Set persistence mode: 0/DISABLED, 1/ENABLED
-e, --ecc-config= Toggle ECC support: 0/DISABLED, 1/ENABLED
-p, --reset-ecc-errors= Reset ECC error counts: 0/VOLATILE, 1/AGGREGATE
-c, --compute-mode= Set MODE for compute applications:
0/DEFAULT, 1/EXCLUSIVE_THREAD (DEPRECATED),
2/PROHIBITED, 3/EXCLUSIVE_PROCESS
--gom= Set GPU Operation Mode:
0/ALL_ON, 1/COMPUTE, 2/LOW_DP
-r --gpu-reset Trigger reset of the GPU.
Can be used to reset the GPU HW state in situations
that would otherwise require a machine reboot.
Typically useful if a double bit ECC error has
occurred.
Reset operations are not guarenteed to work in
all cases and should be used with caution.
-vm --virt-mode= Switch GPU Virtualization Mode:
Sets GPU virtualization mode to 3/VGPU or 4/VSGA
Virtualization mode of a GPU can only be set when
it is running on a hypervisor.
-lgc --lock-gpu-clocks= Specifies <minGpuClock,maxGpuClock> clocks as a
pair (e.g. 1500,1500) that defines the range
of desired locked GPU clock speed in MHz.
Setting this will supercede application clocks
and take effect regardless if an app is running.
Input can also be a singular desired clock value
(e.g. <GpuClockValue>). Optionally, --mode can be
specified to indicate a special mode.
-m --mode= Specifies the mode for --locked-gpu-clocks.
Valid modes: 0, 1
-rgc --reset-gpu-clocks
Resets the Gpu clocks to the default values.
-lmc --lock-memory-clocks= Specifies <minMemClock,maxMemClock> clocks as a
pair (e.g. 5100,5100) that defines the range
of desired locked Memory clock speed in MHz.
Input can also be a singular desired clock value
(e.g. <MemClockValue>).
-rmc --reset-memory-clocks
Resets the Memory clocks to the default values.
-lmcd --lock-memory-clocks-deferred=
Specifies memClock clock to lock. This limit is
applied the next time GPU is initialized.
This is guaranteed by unloading and reloading the kernel module.
Requires root.
-rmcd --reset-memory-clocks-deferred
Resets the deferred Memory clocks applied.
-ac --applications-clocks= Specifies <memory,graphics> clocks as a
pair (e.g. 2000,800) that defines GPU's
speed in MHz while running applications on a GPU.
-rac --reset-applications-clocks
Resets the applications clocks to the default values.
-pl --power-limit= Specifies maximum power management limit in watts.
Takes an optional argument --scope.
-sc --scope= Specifies the device type for --scope: 0/GPU, 1/TOTAL_MODULE (Grace Hopper Only)
-cc --cuda-clocks= Overrides or restores default CUDA clocks.
In override mode, GPU clocks higher frequencies when running CUDA applications.
Only on supported devices starting from the Volta series.
Requires administrator privileges.
0/RESTORE_DEFAULT, 1/OVERRIDE
-am --accounting-mode= Enable or disable Accounting Mode: 0/DISABLED, 1/ENABLED
-caa --clear-accounted-apps
Clears all the accounted PIDs in the buffer.
--auto-boost-default= Set the default auto boost policy to 0/DISABLED
or 1/ENABLED, enforcing the change only after the
last boost client has exited.
--auto-boost-permission=
Allow non-admin/root control over auto boost mode:
0/UNRESTRICTED, 1/RESTRICTED
-mig --multi-instance-gpu= Enable or disable Multi Instance GPU: 0/DISABLED, 1/ENABLED
Requires root.
-gtt --gpu-target-temp= Set GPU Target Temperature for a GPU in degree celsius.
Requires administrator privileges
[plus optional]
-i, --id= Target a specific GPU.
-eow, --error-on-warning Return a non-zero error for warnings.
UNIT MODIFICATION OPTIONS:
-t, --toggle-led= Set Unit LED state: 0/GREEN, 1/AMBER
[plus optional]
-i, --id= Target a specific Unit.
SHOW DTD OPTIONS:
--dtd Print device DTD and exit.
[plus optional]
-f, --filename= Log to a specified file, rather than to stdout.
-u, --unit Show unit, rather than device, DTD.
--debug= Log encrypted debug information to a specified file.
Device Monitoring:
dmon Displays device stats in scrolling format.
"nvidia-smi dmon -h" for more information.
daemon Runs in background and monitor devices as a daemon process.
This is an experimental feature. Not supported on Windows baremetal
"nvidia-smi daemon -h" for more information.
replay Used to replay/extract the persistent stats generated by daemon.
This is an experimental feature.
"nvidia-smi replay -h" for more information.
Process Monitoring:
pmon Displays process stats in scrolling format.
"nvidia-smi pmon -h" for more information.
TOPOLOGY:
topo Displays device/system topology. "nvidia-smi topo -h" for more information.
DRAIN STATES:
drain Displays/modifies GPU drain states for power idling. "nvidia-smi drain -h" for more information.
NVLINK:
nvlink Displays device nvlink information. "nvidia-smi nvlink -h" for more information.
C2C:
c2c Displays device C2C information. "nvidia-smi c2c -h" for more information.
CLOCKS:
clocks Control and query clock information. "nvidia-smi clocks -h" for more information.
ENCODER SESSIONS:
encodersessions Displays device encoder sessions information. "nvidia-smi encodersessions -h" for more information.
FBC SESSIONS:
fbcsessions Displays device FBC sessions information. "nvidia-smi fbcsessions -h" for more information.
GRID vGPU:
vgpu Displays vGPU information. "nvidia-smi vgpu -h" for more information.
MIG:
mig Provides controls for MIG management. "nvidia-smi mig -h" for more information.
COMPUTE POLICY:
compute-policy Control and query compute policies. "nvidia-smi compute-policy -h" for more information.
BOOST SLIDER:
boost-slider Control and query boost sliders. "nvidia-smi boost-slider -h" for more information.
POWER HINT: power-hint Estimates GPU power usage. "nvidia-smi power-hint -h" for more information.
BASE CLOCKS: base-clocks Query GPU base clocks. "nvidia-smi base-clocks -h" for more information.
CONFIDENTIAL COMPUTE:
conf-compute Control and query confidential compute. "nvidia-smi conf-compute -h" for more information.
GPU PERFORMANCE MONITORING:
gpm Control and query GPU performance monitoring unit. "nvidia-smi gpm -h" for more information.
PCI:
pci Display device PCI information. "nvidia-smi pci -h" for more information.
Please see the nvidia-smi(1) manual page for more detailed information.
And there is my memory usage:
