【RDMA】mlxlink检查和调试连接状态及相关问题--驱动工具

bandaoyu2024-09-23 20:01

简介

mlxlink工具用于检查和调试连接状态及相关问题。该工具可以用于不同的链路和电缆（包括被动、电动、收发器和背板）。

属于mft工具套件的一个工具，固件工具 Firmware Tools (MFT):https://blog.csdn.net/bandaoyu/article/details/142427579

警告

为了确保mlxlink正常工作，请确保将固件版本更新到最新版本。

mlxlink是为具备适当技术背景的高级用户设计的。

请不要使用mlxlink通过（"--port_state dn"）标志来禁用主机与无管理交换机之间的端口连接。

mlxlink的错误、警告和说明将打印在标准错误控制台上。

为新设备（NVIDIA ConnectX-6及以上，NVIDIA Quantum交换机及以上）设置速率（50GbE、100GbE和200GbE），需要指定通道数量：mlxlink -d <dev> --speeds [50G_2X | 50G_1X | 100G_2X | 100G_4X | 200G_4X ]。

官网说明：https://docs.nvidia.com/networking/display/mftv421/mlxlink+utility

功能

  • 监控功能：

    • 错误计数器和比特错误率信息
    • 链路等级
    • 前向纠错信息
    • 一般端口信息
    • SerDes 参数
    • 设备信息
    • PCIe

  • 配置功能：

    • 重置端口
    • 速度
    • SerDes 参数
    • 前向纠错
    • 循环回路

示例

获取 <设备>，<端口编号> 的信息：状态,带宽，固件版本，mft版本等

mlxlink -d <device> -p <port_number>

mlxlink -d mlx5_bond_0 -p 1 /mlxlink -d 81:00.0 -p 1

获取 <设备>，<端口编号> 和比特错误率计数器的信息。

mlxlink -d <device> -p <port_number> -c

mlxlink -d mlx5_bond_0 -p 1 /mlxlink -d 81:00.0 -p 1

（编辑中.....）

mlxlink 用法

语法：

mlxlink [OPTIONS]

选项:

|-----------------------------|---------------------------------------------------------------------------|
| -h |--help | Display help message. |
| -v |--version | Display version info. |
| -d |--device <device> | Perform operation for a specified mst device |
| -p |--port <port_number> | Port Number |
| --port_type <port_type> | Port Type [NETWORK(Default)/PCIE/OOB] |
| --depth <depth> | Depth level of the DUT of some hierarchy (valid for PCIe port type only) |
| --pcie_index <pcie_index> | PCIe index number (Internal domain index) (valid for PCIe port type only) |
| --node <node> | The node within each depth (valid for PCIe port type only) |
| --json | Print the output in JSON format |

Queries:

|-----------------------------------|-----------------------------------------------------------------------------------------------------------|
| --show_links | Show valid PCIe links (valid for PCIe port type only) |
| -m |--show_module | Show Module Info |
| -c |--show_counters | Show Physical Counters and BER Info |
| -e |--show_eye | Show Eye Opening Info |
| --show_fec | Show FEC Capabilities |
| --show_serdes_tx | Show Transmitter Info |
| --show_tx_group_map <group_num> | Display all label ports mapped to group <group_num> (for NVIDIA Spectrum-2 and NVIDIA Quantum devices). |
| --show_device | General Device Info |
| --show_ber_monitor | Show BER Monitor Info. Note: The flag is not supported in HCAs. |
| --show_external_phy | Show External PHY Info Note: The flag is supported in NVIDIA Spectrum switch systems only. |

Commands:

|---|---|---|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| -a |--port_state <port_state> |||| Configure Port State [UP(up)/DN(down)/TG(toggle)] |
| -s |--speeds <speeds> |||| Configure Speeds [speed1,speed2,...] |
| | --link_mode_force ||| Configure Link Mode Force (Disable AN) |
| -l |--loopback <loopback> |||| Configure Loopback Mode [NO(no loopback)/RM(phy remote Rx-to-Tx loopback)/PH(internal phy Tx-to-Rx loopback)/EX(external loopback connector needed)] |
| -k |--fec <fec_override> |||| Configure FEC [AU(Auto)/NF(No-FEC)/FC(FireCode FEC)/ RS(RS-FEC)]/LL(LL-RS-FEC)/DF-RS(Interleaved_RS-FEC)/DF-LL(Interleaved_LL_RS-FEC)] |
| | --fec_speed <fec_speed> ||| Speed to Configure FEC [100G/50G/25G/...] (Default is Active Speed) |
| --serdes_tx <params> |||| Configure Transmitter Parameters [polarity,ob_tap0,...] |
| | --serdes_tx_lane <transmitter_lane> ||| Transmitter Lane to Set (Optional - Default All Lanes) |
| | --database ||| Save Transmitter Configuration for Current Speed Permanently (Optional) |
| | --tx_params_override ||| Set the parameters according to Data Base only, otherwise it will be set according to the best possible configuration chosen by the system (e.g. KR-startup) (Optional) |
| --tx_group_map <group_num> |||| Map ports to group <group_num> (for NVIDIA Spectrum-2 and NVIDIA Quantum devices) |
| | --ports <ports> ||| Ports to be mapped [1,2,3,4..] |
| --test_mode <prbs_mode> |||| Physical Test Mode Configuration [EN(enable)/DS(disable)/TU(perform tuning)] |
| | --rx_prbs <rx_prbs_mode> ||| RX PRBS Mode [PRBS31(Default)/PRBS7/...] (Optional - Default PRBS31) |
| | --tx_prbs <tx_prbs_mode> ||| TX PRBS Mode [PRBS31(Default)/PRBS7/...] (Optional - Default PRBS31) |
| | --rx_rate <rx_lane_rate> ||| RX Lane Rate [EDR(Default)/25G/10G/...] (Optional - Default 25G) |
| | --tx_rate <tx_lane_rate> ||| TX Lane Rate [EDR(Default)/25G/10G/...] (Optional - Default 25G) |
| | --invert_tx_polarity ||| PRBS TX polarity inversion (Optional - Default No Inversion) |
| | --invert_rx_polarity ||| PRBS RX polarity inversion (Optional - Default No Inversion) |
| | --lanes ||| PRBS lanes to set (one or more lane separated by comma)[0,1,2,...] Optional: Default all lanes |
| -b |--ber_collect <csv_file> |||| Port Extended Information Collection [CSV File] |
| --amber_collect <csv_file> |||| AmBER Port Extended Information Collection For 16nm Products and Later [CSV File] |
| --ber_limit <limit_criteria> |||| BER Limit Criteria [Nominal(Default)/Corner/Drift] (Optional - Default Nominal) |
| | --iteration <iteration> ||| Iteration Number of BER Collection |
| --pc |||| Clear Counters |
| --set_external_phy |||| Set External PHY Note: The flag is supported in NVIDIA Spectrum switch systems only. |
| | --twisted_pair_force_mode <twisted_pair_force_mode> ||| Twisted Pair Force Mode [MA(Master)/SL(Slave)] |
| --cable |||| Perform operations on the cables |
| | --dump ||| Dump cable pages in raw format |
| | --ddm ||| Get cable Digital Diagnostic Monitoring information |
| | --read ||| Perform read operation from specific page |
| || --length <length> || Length of data to read in bytes (Optional - Default 1 byte) |
| || --page <pageNum> || Specific page number to read/write |
| || --offset <offset> || Specific page offset to read/write |
| | --write <bytes> ||| Perform write operation with specific data (list of bytes, separated by ',') |
| | --prbs_select <side> ||| Module PRBS test mode side selector [MEDIA, HOST] |
| || --prbs_mode <cmd> || Perform PRBS test mode on the Module [EN(Enable),DS(Disable)] |
| ||| --generator_pattern <pattern> | Set PRBS generator pattern [PRBS31(default),PRBS23,PRBS7,PRBS11,PRBS9,PRBS13,SSPR,SSPRQ] |
| ||| --swap_generator | Enable PAM4 MSB <-> LSB generator swapping (Optional) |
| ||| --invert_generator | Enable PRBS generator inversion (Optional) |
| ||| --generator_lanes <lanes> | PRBS generator lanes to set (one or more lane separated by comma)[0,1,2,3,4,5,6,7] (Optional - Default all lanes) |
| ||| --checker_pattern <pattern> | Set PRBS checker pattern [PRBS31(default),PRBS23,PRBS7,PRBS11,PRBS9,PRBS13,SSPR,SSPRQ] |
| ||| --swap_checker | Enable PAM4 MSB <-> LSB checker swapping (Optional) |
| ||| --invert_checker | Enable PRBS checker inversion (Optional) |
| ||| --checker_lanes <lanes> | PRBS checker lanes to set (one or more lane separated by comma)[0,1,2,3,4,5,6,7] (Optional - Default all lanes) |
| ||| --lane_rate <rate> | Set PRBS checker and generator lane rate [HDR(50G)(default),1.25G,SDR(2.5G),10.3125G,FDR(14G),EDR(25G),NDR(100G)] |
| || --show_diagnostic_info || Show PRBS diagnostic counters information |
| || --clear_diagnostic_info || Clear PRBS diagnostic counters |
| | --control_parameters ||| Show Module Control Parameters |
| || --tx_equalization <value> || Set Module Tx Input Equalization in dB [NE(No Equalization),1,2,3,4,5,6,7,8, 9,10,11,12] |
| || --rx_emphasis <value> || Set Module RX Output Emphasis in dB. for CMIS, pre-emphasis value will be set [NE(No Equalization),0.5,1,1.5,2,2.5,3,3.5,4,5,6,7] |
| || --rx_post_emphasis <value> || Set Module Rx Post Emphasis in dB [NE(No Equalization),1,2,3,4,5,6,7] |
| || --rx_amplitude <value> || Set Module Rx Output Amplitude [0(100-400mV),1(300-600mV),2(400-800mV),3(600-1200mV)] |
| --margin |||| Read the SerDes eye margins per lane |
| | --measure_time <time> ||| Measure time in seconds for single eye [10, 30, 60, 90, 120, 240, 480, 600 and 900] (Optional - Default 60 for PCIe and 30 for Network ports) |
| | --eye_select <eye_sel> ||| Eye selection for PAM4 [UP, MID, DOWN, ALL] (Default ALL) |
| | --lane <lane_index> ||| Run eye for specific lane index (Default all lanes) |
| --rx_error_injection |||| Enable the RX link deterioration |
| | --mixer_offset0 <value> ||| Fine change to the center of the eye [0x0 to 0x7ff] |
| | --mixer_offset1 <value> ||| Coarse change to the center of the eye [0x0 to 0x3ff] |
| | --show_mixers_offset ||| Show mixer offset 0 and mixer offset 1 |
| --rx_fec_histogram |||| Provide histogram of FEC errors. The result is divided to bins. Each bin is holding different number of errored bit within FEC protected block |
| | --show_histogram ||| Show FEC errors histogram |
| | --clear_histogram ||| Clears FEC errors histograms |
| --yes |||| Non-interactive mode, answer yes to all questions |

Examples:

Get info of <device>, <port_number>:

复制代码

mlxlink -d <device> -p <port_number>

Get info of <device>, <port_number> and BER Counters:

复制代码

mlxlink -d <device> -p <port_number> -c

Get info of <device>, <port_number> and Transmitter Parameters:

复制代码

mlxlink -d <device> -p <port_number> --show_serdes_tx

Configure Port State:

复制代码

mlxlink -d <device> -p <port_number> --port_state UP

Configure Port Speeds:

复制代码

mlxlink -d <device> -p <port_number> --speeds 25G,50G,100G

Configure FEC:

复制代码

mlxlink -d <device> -p <port_number> --fec RS

Configure Port for Physical Test Mode:

复制代码

mlxlink -d <device> -p <port_number> --test_mode EN (--rx_prbs PRBS31 --rx_rate 25G --tx_prbs PRBS7 --tx_rate 10G --invert_rx_polarity --invert_tx_polarity)

Perform PRBS Tuning:

复制代码

mlxlink -d <device> -p <port_number> --test_mode TU

Warning

RX and TX lane rates for new devices includes the PAM4 speeds (50G_1X and 100G_2X)eg: mlxlink -d <device> --test_mode EN --rx_rate [normal speeds | 50G_1X | 100G_2X] --tx_rate [normal speeds | 50G_1X | 100G_2X]

Warning

The PRBS pattern that is configured in PAM4 rates is PRBSQ.

Cable operations:

复制代码

mlxlink -d <device> --cable [Options]

Dump cable EEPROM pages:

复制代码

mlxlink -d <device> --cable --dump

Get cable DDM info:

复制代码

mlxlink -d <device> --cable --ddm

Read from cable:

复制代码

mlxlink -d <device> --cable --read --page <page number> --offset <bytes offset> --length <number of bytes>

Write to cable:

复制代码

mlxlink -d <device> --cable --write <bytes separated by comma> --page <page number> --offset <bytes offset>

Configure Transmitter Parameters (on lane, to database):

复制代码

mlxlink -d <device> -p <port_number> --serdes_tx <polar- ity>,<ob_tap0>,<ob_tap1>,<ob_tap2>,<ob_bias>,<ob_preemp_mode>,<ob_reg>,<ob_leva> (--serdes_tx_lane <lane number>) (--database)

Configure Transmitter Parameters for 16nm devices:

复制代码

mlxlink -d <device> -p <port_number> --serdes_tx <pre_2_tap>,<pre_tap>,<main_tap>,<post_tap>,<ob_m2lp>,<ob_amp>

Getting PCIe links info:

复制代码

mlxlink -d /dev/mst/mt41682_pciconf0 --port_type PCIE --show_links Valid PCIe Links ---------------- : depth, pcie_index, node, port Link ``1`` : ``3``, ``0``, ``0``, ``60`` Link ``2`` : ``3``, ``0``, ``1``, ``61`` Link ``3`` : ``3``, ``0``, ``2``, ``62`` ..

To query information for a specific link, the depth, pcie_index and node for the link must be specified:

复制代码

mlxlink -d /dev/mst/mt41682_pciconf0 --port_type PCIE --depth ``3`` --pcie_index ``0`` --node ``1`` --show_serdes_tx --show_eye PCIe Operational (Enabled) Info ------------------------------- Depth, pcie index, node : ``3``, ``0``, ``1`` Link Speed Active (Enabled) : 8G-Gen ``3`` (16G-Gen ``4``) Link Width Active (Enabled) : 2X (16X) EYE Opening Info (PCIe) ----------------------- Physical Grade : ``84``, ``84`` Height Eye Opening [mV] : ``1194``, ``1194`` Phase Eye Opening [psec] : ``84``, ``84`` Serdes Tuning Transmitter Info (PCIe) ------------------------------------- Serdes TX parameters : Pol ,tap0 ,tap1 ,tap2 ,bias ,preemp_mode ,reg ,leva Lane ``0`` : ``0`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``1`` : ``1`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``2`` : ``0`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``3`` : ``1`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``4`` : ``0`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``5`` : ``1`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``6`` : ``0`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9`` Lane ``7`` : ``1`` ,``21`` ,``92`` ,``7`` ,``15`` ,``1`` ,``10`` ,``9``

To print the output in JSON format:

复制代码

mlxlink -d <device> --show_module --json

To show ports group map (for NVIDIA Quantum and NVIDIA Spectrum-2):

复制代码

mlxlink --d<device> --show_tx_group_map ``0

To assign ports to a specific group on NVIDIA Quantum and NVIDIA Spectrum-2

复制代码

mlxlink --d <device> --tx_group_map ``1`` --ports ``1``,``2``,``3``,``5``,``4``,``8``,``7``,``8``,``9``,``10``,``11

To show histogram of FEC errors:

复制代码

mlxlink -d /dev/mst/mt4125_pciconf0 --rx_fec_histogram --show_histogram

To clear histogram:

复制代码

mlxlink -d /dev/mst/mt4125_pciconf0 --rx_fec_histogram --clear_histogram

Margin Scan Tool

The margin scan tool is used for scanning PCIe or Network ports [EDR\25G or HDR\PAM4 speeds].

Warning

If the margin scan fails with this message (Eye scan not completed), perform a reboot and run the scan again.

To enable the margin scan with measure time 10 seconds:

复制代码

mlxlink --d <device> --port_type PCIE --margin --measure_time ``10

To enable the margin scan for Multi-host or Socket Direct systems through:

  • depth, pcie_index and node:

    复制代码

    Mlxlink --d <device> --port_type PCIE --depth ``0`` --pcie_index ``1`` --node ``0`` --margin --measure_time ``30

  • The local port (it can be shown by the --show_links command):

    复制代码

    Mlxlink --d <device> --port_type PCIE --port ``1`` --margin --measure_time ``10

RX Error Injection

Allows modifying the Eye Center capability by changing the mixer_offset0 (fine change) and mixer_offset1 (coarse change) flags for 28nm products to produce RX errors.

Flags Usage

  • To change the values of mixers:

    复制代码

    mlxlink -d /dev/mst/mt4117_pciconf0 --rx_error_injection --mixer_offset0 ``0x200`` --mixer_offset1 ``0x305

    Warning

    Modifying mixer_offset0 and mixer_offset1 flags can change the Eye Center and might cause link degradation.

  • To query the mixers values:

    复制代码

    mlxlink -d /dev/mst/mt4117_pciconf0 --rx_erro r_injection --show_mixers_offset

Module PRBS Test Mode

The module PRBS test mode can be performed by using the new flags under the --cable command.

Note

  • This feature supports Active/Optical CMIS modules only

  • Either the media or host side can run with PRBS mode

  • To enable the PRBS test mode, the module should be plugged in and active

Enabling\Disabling The Module PRBS Test Mode

To enable the module PRBS test mode, the side of the module should be selected using --prbs_select flag after providing the --cable flag, it can be either the HOST or the MEDIA side, then the --prbs_mode <EN\DS> can be used to enable or disable the PRBS test mode process.

E.g: the following command will enable the PRBS test mode on the HOST side of the module:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --port ``3`` --cable --prbs_select HOST --prbs_mode EN

The command above will put the HOST side of the module in PRBS test mode with default Checker and Generator parameters.

The Checker and Generator parameters can be overridden while enabling the PRBS test mode according to their related flags in the help menu:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --port ``3`` --cable --prbs_select HOST --prbs_mode EN --checker_pattern PRBS13 --invert_checker --generator_pattern PRBS31 --swap_generator --lane_rate HDR

To disable the PRBS test mode, the following command can be executed:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --port ``3`` --cable --prbs_select HOST --prbs_mode DS

PRBS Diagnostic Counters Information

After performing the PRBS test mode, the module counters can be quired by using the following command:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --port ``3`` --cable --prbs_select HOST --show_diagnostic_info

The module PRBS test mode counters can be cleared by using the following command, which will clear the diagnostic counters on the HOST side only:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --port ``3`` --cable --prbs_select HOST --clear_diagnostic_info

Module Control Parameters

Some of the module parameters can be controlled by mlxlink after providing the --control_paramenter flag, which can be executed under --cable flag.

The possible parameters can be controlled as follows:

  • Reading and configuring Tx Equalization

  • Reading and configuring Rx Emphasis (PreCursor & PostCursor)

  • Reading and configuring Rx Amplitude

Note

  • To apply the changes,the link should be disabled first

  • After configuring a new parameter, the link should be raised again to allow the firmware to load the new configuration

  • Cable control parameters are valid for active\optical modules only

Query, Configure The Module Control Parameters

To query the currently configured module control parameters, the --control_parameters flag can be used under --cable flag as follows:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --cable --control_parameters ... Module Control Parameters ------------------------- TX Equalization : 1dB RX Emphasis (pre) : ``2``.5dB RX Post Emphasis : No Equalization RX Amplitude : ``600``-``1200`` mV (P-P)

To configure the module control parameters, the following command can be executed:

复制代码

mlxlink -d /dev/mst/mt53104_pciconf0 --cable --control_parameters --tx_equalization ``2`` --rx_amplitude ``1``

Tool Usage with NIC vs. Switch (-p Flag)

When using mlxlink tool with NIC, notice that the "label_port" flag -p should not be used. To address different ports please use different mst devices.

For example:

To address port 1 in ConnectX-4 use:

复制代码

mlxlink -d /dev/mst/mt4115_pciconf0

To address port 2 use:

复制代码

mlxlink -d /dev/mst/mt4115_pciconf0.``1

Warning

  • Any mlxlink command for switch should include the "-p" flag to address the specific port in the switch.

  • When working with the NIC, if an MTUSB is used for communication with the NVIDIA NIC, to address port 2, use mlxlink -d /dev/mst/mt4115_pciconf0 --gvmi_address<0xAddress>.

Tool Usage on NVIDIA Quantum HDR Switch Systems with Split Ports

Using mlxlink on NVIDIA Quantum HDR based switch systems split ports if the split port number is not provided by the ibdiagnet tool:

复制代码

mlxlink -d lid-<LID> -p <formula>

Formula:

In case of 2X port:

  • 1- port_num = round_down[(Iblinkinfo_port_num + 1)*0.5]

  • 2- if (Iblinkinfo_port_num + 1) modulo 2 =1 then append '/2' to port_num

In case of 4X port, use only item #1 above

Example:

复制代码

43`` ``23``[ ] ==( 2X ``53.125`` Gbps Active/ LinkUp)==> mlxlink -d lid-``43`` -p ``12`` ``43`` ``24``[ ] ==( 2X ``53.125`` Gbps Active/ LinkUp)==> mlxlink -d lid-``43`` -p ``12``/``2

Tool Usage on NVIDIA Quantum-2 NDR Switch Systems

In NVIDIA Quantum-2 NDR switch generation there are 32-OSFP cages (8x), where each one holds 2 (4x) ports instead of 1, and each port can be accessed by providing the cage number and the port in the cage "Cage/Port".

复制代码

mlxlink -d <mst deivce> -p <Cage>/<Port>

If the split profile is ready, then we can access the split ports by providing the number of split to the port flag, e.g.:

  • To access the main port of 15/2

    复制代码

    mlxlink -d <mst deivce> -p ``15``/``2

  • To access the split port of 15/2

    复制代码

    mlxlink -d <mst deivce> -p ``15``/``2``/``2

PCIe

For PCIe link speed and width use the following flag: --port_type PCIE

复制代码

PCIe Operational (Enabled) Info ------------------------------- Depth, pcie index, node : [Depth, pcie index, node] Link Speed Active (Enabled) : [Freq -- Gen] Link Width Active (Enabled) : [Width]

PCIe Switch

For NVIDIA ConnectX-5 and newer devices, the PCIe interface can be configured for PCIe switch. When the PCIe switch is enabled, the Depth, pcie_index and node parameters are needed to specify the PCIe port where the requested information (such as counters or eye info) is gathered from.

Parameters Description
Depth This defines the number of layers from the Root Complex to the specific port. * For NVIDIA ConnectX adapter cards multi-host mode, the Depth should be set to 0. * For NVIDIA BlueField/BlueField-2 JBoF, the Depth should be set to 3.
Pcie_index This defines the root complex ID or host index. * For NVIDIA ConnectX adapter cards multi-host mode, the pcie_index is the host index (0 -- 3). * For NVIDIA BlueField/BlueField-2 JBoF, the pcie_index is always 0.
Node This defines the specific pcie port. * For NVIDIA ConnectX adapter cards multi-host mode, the node is always 0 for each host_index. * For NVIDIA BlueField JBoF mode, this is from 0x0 -- 0xF, accounting for up to 16 possible ports for BlueField JBoF. * For NVIDIA BlueField-2, this is 0x0 -- 0x7. Note: For NVIDIA BlueField/BlueField-2 SmartNIC mode, the PCIe link information can only be gathered from the external host. The PCIe interface status cannot be retrieved from the Arm side. When retrieving the PCIe link information from the external host, there is no need to specify the depth, pcie_index and node.

Example: NVIDIA BlueField JBoF Mode

复制代码

# mlxlink -d /dev/mst/mt41682_pciconf0 --port_type pcie --depth ``3`` --pcie_index ``0`` --node ``4`` -c PCIe Operational (Enabled) Info ------------------------------- Depth, pcie index, node : ``3``, ``0``, ``4`` Link Speed Active (Enabled) : 8G-Gen ``3`` (16G-Gen ``4``) Link Width Active (Enabled) : 2X (2X) Management PCIe Timers Counters Info ------------------------------------ dl down : ``0`` Management PCIe Performance Counters Info ----------------------------------------- RX Errors : ``0`` TX Errors : ``0`` CRC Error dllp : ``0`` CRC Error tlp : ``0

For PCIe counters information use the following flag: --port_type PCIE --c

复制代码

Management PCIe Timers Counters Info ------------------------------------ dl down : [link down counter] Management PCIe Performance Counters Info ----------------------------------------- RX Errors : [Rx Errors] TX Errors : [Tx Errors] CRC Error dllp : [CRC Errors dllp] CRC Error tlp : [CRC Errors tlp]

  • RX errors: indicate number of transitions to recovery due to Framing errors and CRC (dlp and tlp) errors.

  • TX errors: indicate number of transitions to recovery due to EIEOS and TS errors.

  • CRC Error dllp: indicate CRC error in Data Link Layer Packets

  • CRC Error tlp: indicate CRC error in Transaction Layer Packet

Example:

复制代码

# mlxlink -d /dev/mst/mt4123_pciconf0 --port_type PCIE -c PCIe Operational (Enabled) Info ------------------------------- Depth, pcie index, node : ``0``, ``0``, ``0`` Link Speed Active (Enabled) : 16G-Gen ``4`` (16G-Gen ``4``) Link Width Active (Enabled) : 16X (16X) Management PCIe Timers Counters Info ------------------------------------ dl down : ``3`` Management PCIe Performance Counters Info ----------------------------------------- RX Errors : ``0`` TX Errors : ``16`` CRC Error dllp : ``0`` CRC Error tlp : ``0

For PCIe link physical grade and eye opening information use the following flag: --port_type PCIE --e

复制代码

EYE Opening Info (PCIE) ------------------------ Physical Grade : [Grade0, Grade1, Grade2, Grade3, Grade4, Grade5, Grade6, Grade7, Grade8, Grade9, Grade10, Grade11, Grade12, Grade13, Grade14, Grade15] Height Eye Opening [mV] : [Height0, Height1, Height2, Height3, Height4, Height5, Height6, Height7, Height8, Height9, Height10, Height11, Height12, Height13, Height14, Height15] Phase Eye Opening [psec] : [Phase0, Phase1, Phase2, Phase3, Phase4, Phase5, Phase6, Phase7, Phase8, Phase9, Phase10, Phase11, Phase12, Phase13, Phase14, Phase15]

Example:

复制代码

# mlxlink -d /dev/mst/mt4123_pciconf0 --port_type PCIE -e PCIe Operational (Enabled) Info ------------------------------- Depth, pcie index, node : ``0``, ``0``, ``0`` Link Speed Active (Enabled) : 16G-Gen ``4`` (16G-Gen ``4``) Link Width Active (Enabled) : 16X (16X) EYE Opening Info (PCIe) ----------------------- Physical Grade : ``57279``, ``56340``, ``59340``, ``61824``, ``55140``, ``60501``, ``61530``, ``57392``, ``61573``, ``58930``, ``62752``, ``60421``, ``57188``, ``59796``, ``60066``, ``60847`` Height Eye Opening [mV] : ``292``, ``288``, ``314``, ``325``, ``278``, ``310``, ``319``, ``299``, ``316``, ``318``, ``343``, ``323``, ``310``, ``311``, ``335``, ``318`` Phase Eye Opening [psec] : ``30``, ``30``, ``30``, ``30``, ``30``, ``30``, ``30``, ``30``, ``30``, ``28``, ``28``, ``28``, ``28``, ``30``, ``28``, ``30

Pass / Fail Criteria

SLRED (ConnectX-6/ConnectX-6 Dx/ConnectX-6 Lx)

复制代码

mlxlink -d [device] --port_type PCIE --margin

Gen3

|--------------------------|-----------------------|
| Gen3 ||
| Eye Grade | Figure of Merit (FOM) |
| 0 < Eye Grade < 700 | FAIL |
| 700 < Eye Grade < 2300 | gray area |
| 2300 < Eye Grade | PASS |

Gen4

|-------------------------|-----------|
| Gen4 ||
| Eye Margin | FOM |
| 0 < Eye Grade < 150 | FAIL |
| 150 < Eye Grade < 400 | gray area |
| 400 < Eye Grade | PASS |

© Copyright 2023, NVIDIA. Last updated on Oct 11, 2023.

相关推荐
小林熬夜学编程7 分钟前
【Linux系统编程】第四十九弹---日志系统构建指南：从基础结构到时间处理与Log类实现
linux·运维·服务器·c语言·开发语言·c++
zhangshengqiang1688 分钟前
Linux下安装mysql8.0版本
linux·mysql
网络安全King9 分钟前
网络安全设备系列--安全隔离网闸
网络·安全·web安全
程序猿小D20 分钟前
第三百三十节 Java网络教程 - Java网络UDP服务器
java·开发语言·网络·网络协议·udp·多线程
半桶水专家23 分钟前
tcpdump抓取流量包详解
网络·测试工具·tcpdump
linweidong39 分钟前
MariaDB面试题及参考答案
linux·运维·数据库·负载均衡·dba·mariadb·后端面试
杰哥的技术杂货铺2 小时前
Centos 7 安装 Docker 最新版本
linux·docker·centos
jerry-892 小时前
CentOS 7安装SSHFS 实现远程主机目录 挂载为本地目录
linux·运维·centos
Jtti3 小时前
Jtti：如何知晓服务器的压力上限?具体的步骤和方法
运维·服务器
命里有定数3 小时前
Ubuntu问题 - 显示ubuntu服务器上可用磁盘空间 一条命令df -h
服务器·ubuntu·数据集
热门推荐
01(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明02PyTorch机器学习实现液态神经网络03【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总04玄机平台应急响应—webshell查杀05Coze扣子平台完整体验和实践（附国内和国际版对比）06RAG 实践- Ollama+RagFlow 部署本地知识库07【目标跟踪】相机运动补偿08Unity中PICO实现 隔空取物 和 接触抓取物体09Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO1004 - matlab m_map地学绘图工具基础函数 - 设置网格