3

I have a Supermicro server with a X9DRT-HIBFF motherboard installed.

It has on-board Infiniband based on the ConnectX-3 chipset from Mellanox.

Supermicro supply firmware version 2.10.0700 and unfortunately it doesn't support SR-IOV, it's too old. Although the chipset and BIOS does support this.

Newer firmware from Mellanox enables this feature. Mellanox themselves do supply firmware for some OEM partners, but not Supermicro.

ibv_devinfo shows:

$ sudo ibv_devinfo
hca_id: mlx4_0
    transport:          InfiniBand (0)
    fw_ver:             2.10.700
    node_guid:          0025:90ff:ff17:5724
    sys_image_guid:         0025:90ff:ff17:5727
    vendor_id:          0x02c9
    vendor_part_id:         4099
    hw_ver:             0x0
    board_id:           SM_2191000001000
    phys_port_cnt:          1
        port:   1
            state:          PORT_DOWN (1)
            max_mtu:        4096 (5)
            active_mtu:     4096 (5)
            sm_lid:         0
            port_lid:       0
            port_lmc:       0x00

So the question is, can Mellanox firmware be burned to this OEM board? If so, how would I select which one to burn?

Looking at this firmware table, the corresponding firmware links to a slightly different ZIP file. The descriptions are nearly all identical with the same vendor part id of 4099. The only difference between them is the name and PSID. My PSID being: SM_2191000001000 Which is different from any of them as expected because it's from Supermicro.

hookenz
  • 14,132
  • 22
  • 86
  • 142

1 Answers1

1

  1. Download the firmware and extract it

  2. Download some tools from Mellanox and install the corresponding deb file to get the "mlxburn" utility in your Ubuntu installation.

    wget http://www.mellanox.com/downloads/MFT/mft-3.1.0-8.tgz
tar xzvf mft-3.1.0-8.tgz
cd mft-3.1.0-8/DEBS/
dpkg -i mft-3.1.0-8.amd64.deb

  3. Extract the ini file needed to generate the new firmware from your current firmware. This will include the proper PSID.

    mstflint -d $PCIBUS dc > myinifile.ini

Change the $PCIBUS variable to the PCI bus where your card is connected on (i.e. 04.00.0)

  1. Add the "sriov_en = true" line under the [HCA] section fo the ini file

  2. Generate your firmware like this (I tested this command for ConnectX2):

    mlxburn -fw fw-ConnectX2-rel.mlx -exp_rom FlexBoot-3.4.112/ConnectX_Flexboot_4099_ROM-3.4.112.mrom -conf myinifile.ini -wrimage fw-ConnectX2-rel-sriov-enabled.bin

  3. Install the new firmware in your card

     mlxfwmanager -u -d $PCIBUS -i fw-ConnectX2-rel-sriov-enabled.bin -f
Vangelis Tasoulas
  • 196
  • 1
  • 7
  • I ended up doing this process on some other nodes. Although the exact ini file wasn't available. I discovered the right one that had a different PSID and changed it to match. Then applied the update. mlxburn can actually apply it too. – hookenz Oct 04 '13 at 22:05
  • Closest thing to a correct answer which I did on another hose where the firmware wasn't available for download. Thanks. – hookenz Oct 04 '13 at 22:06