Dual-port QSFP QDR IB/10GbE PCIe 2.0 (FC: 5283, 5285)

Document Revision History	Description
11/14/2013	Updated readme with latest microcode - 000200091316
04/10/2013	Updated the Linux instruction in Section 5.0
03/14/2012	Creating Readme file with latest microcode for this adapter- 000200091000

2.0 General information

This Readme file is intended to give directions on how to update the microcode found on the Dual-port QSFP QDR IB/10GbE PCIe 2.0 (FC: 5283, 5285).

3.0 Installation time

Approximately 20 minutes.

4.0 Machine's Affected:

Feature Code 5283 :

8202-E4B / 8205-E6B, 9125-F2C, 8231-E1C / 8231-E2C, 8202-E4C / 8205-E6C, 8246-L2D / 8246-L2S

Feature Code 5285 :

8202-E4C / 8205-E6C, 9117-MMC / 9179-MHC

5.0 Linux Requirements: Error! Bookmark not defined.

To burn FW for Mellanox ConnectX adapters here are the instructions (for Linux):

1. Find the PCI slot the ConnectX-2 adapter is plugged into. Issue the following command:

lspci | grep Mellanox

For example:

# lspci | grep Mellanox

01:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

This tells us that the adapter is in the PCI slot 0000:01:00.0 which will be needed in the following steps.

2. Start the MFT software suite by running “mst start” at the command line:

#mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

You can ignore the "unable to initialize libusb: -99" error. This occurs on systems without a USB port present.

3. Verify the current Firmware level by running “mstflint -d 01:00.0 -q”. Replace “01:00.0” with the ID found in the lscpi command from step 1.

# mlxburn -d 01:00.0 -q

-I- Image type: ConnectX

-I- FW Version: 2.9.1000

-I- Device ID: 26428

-I- Description: Node Port1 Port2 Sys image

-I- GUIDs: 0002c903000c2d5a 0002c903000c2d5b 0002c903000c2d5c 0002c903000c2d5d

-I- MACs: 0002c90c2d5a 0002c90c2d5b

-I- VSD:

-I- PSID: IBM0FA0000009

This command reveals the current version of the FW; which is 2.9.1000 in this particular case. If the version is less than 2.9.1316, please update.

4. Download the firmware RPM file to the destination machine.

Use this method to download the new microcode to a Linux system:

NOTE: The instructions that follow are specific Linux commands. Linux commands are CASE (lower and upper) SENSITIVE, and must be entered exactly as shown, including filenames.

Transfer the RPM format file to the /tmp directory(using “Save as....”). You will see the filename for the RPM file.
Install rpm on your Linux system by typing:

rpm -ivh /tmp/b3153c6714101504.000200091316.Linux.rpm

The microcode package will install the firmware images in the /lib/firmware directory. If a message is displayed saying the "package <package_name> is already installed", you will need to uninstall the listed rpm package. On the command line type:

rpm -e <package_name>

where <package_name> is the name of the package that was returned in the message. Return to Step 2 and attempt to install the file again.

The microcode file will be installed to the /lib/firmware directory. The file name is：b3153c6714101504.000200091316.

5. Verify the contents of the image before flashing:

ls -l /lib/firmware/b3153c6714101504.000200091316 to verify file size is 671240

sum /lib/firmware/b3153c6714101504.000200091316 to verify Checksum is 18690 for both files listed

6. Update the FW with "flint -d <device id> -i <image filename> burn"

# flint -d 01:00.0 -i b3153c6714101504.000200091316 burn

Current FW version on flash: 2.9.1000

New FW version: 2.9.1316

Burning FW image without signatures - OK

Restoring signature - OK

7. At this point we have burned the new FW on to the Mellanox adapter. However, this FW will not take effect till the system is rebooted.

Reloading the drivers is not enough; it requires the system to be rebooted.

6.0 AIX Requirements:

Basic card function is supported on :

AIX 5L Version V5.3 technology level 5300-12 and later

AIX 6L Version V6.1 technology level 6100-06 and later

AIX 6L Version V6.1 technology level 6100-07 and later

AIX 6L Version V6.1 technology level 6100-08 and later

AIX 7L Version V7.1 technology level 7100-00 and later

AIX 7L Version V7.1 technology level 7100-01 and later

If you are using another release of AIX, ensure that the adapter is supported on that release before you install the adapter. Contact service and support for assistance.

7.0 Determine the Current Microcode Level for AIX:

Before you install the microcode, it is important to determine the microcode level of the Adapter installed in the target system. Use the following instructions to read the ROM level stored in the Adapter's VPD.

1) List the InfiniBand adapters installed in the system by typing:

lscfg -l iba\*

Note: All of the InfiniBand adapters that are installed will be displayed. They will be listed as ibaX, where X is 0, 1, etc.

2) To check the current microcode level for the adapter or controller enter the following command

lsmcode -cd ibax

Where “x” is the instance of the adapter. The command will produce output similar to:

The current microcode level for iba0 is 000200091000

If the ROM Level is less than 000200091316 you should update the microcode.

8.0 Downloading the RPM Format File to the Target Server for AIX
Use this method to download to an AIX system:

NOTE: The instructions that follow are specific AIX commands. AIX commands are CASE (lower and upper) SENSITIVE, and must be entered exactly as shown, including filenames.

1) Make two directories on your AIX system to receive the RPM format file.
Enter: "mkdir /tmp/microcode"

and then create this directory

Enter: "mkdir /tmp/microcode/RPM"

2) Transfer the RPM format file to the /tmp/microcode/RPM directory (using "Save as ..."). Change to that directory, "cd /tmp/microcode/RPM".

You'll see the filename for the RPM file.
"rpm -ihv --ignoreos b3153c6714101504.000200091316.aix.rpm"

3) For AIX: The microcode files will be added to /etc/microcode/.

NOTE:
- "/etc/microcode" is a symbolic link to "/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or file then the user will be prompted for a new location.

9.0 Discovery Tool Microcode CD-ROM creation and download instructions

To obtain information how to burn a CD-ROM and run the Discovery Tool for an AIX or Linux System please go to: http://www14.software.ibm.com/support/customercare/mds/

1) After running the Discovery Tool successfully the "/tmp/microcode/RPM" directory was created and your rpm files are copied from the CD-ROM.

2) Change to that directory, "cd /tmp/microcode/RPM".

3) Unpack the file by executing the instructions below:
Enter the commands:
rpm -ihv --ignoreos b3153c6714101504.000200091316.aix.rpm

4) One microcode file will be copied to "/etc/microcode". The file size and checksum of the microcode image will be verified in Section 9.0.
File Name:
b3153c6714101504.000200091316

10.0 Verifying microcode before download and notes for AIX

Please verify the file size and checksum of the raw microcode files matches what is listed below.

For AIX :
"ls -l /etc/microcode/b3153c6714101504.000200091316" to verify file size is 671240.
"sum /etc/microcode/b3153c6714101504.000200091316" to verify Checksum is 18690.

11.0 Microcode Download Procedure for AIX:

Microcode download must be performed separately for each Adapter under each Logical Partitions (LPAR's).

11.1 Setting up for Microcode download

1) Stop all applications that use this interface/adapter

2) Detach the Adapter’s interface (not always needed, but it is a good practice to do this step.

a) Before detaching the interface, record the ip address and any other pertinent information that was configured on the Adapter. This information may be needed if the microcode update overwrites this section on the Adapter.

11.2 Downloading Microcode to the Adapter

1) At the command line type “diag”
2) Select the "Task Selection" from diagnostics menu.
3) Select "Microcode Tasks” then select “Download Microcode" from the menu.
4) Select all of the PCIE QDR InfiniBand HCA (b3153c67) that need to be updated from the list of devices by using the arrow keys to highlight the entry and pressing "Enter" to mark it. The devices will be displayed as iba#, where # is the number of the device. Press "F7" when you are done marking all the adapters you want to flash.
5) If a source selection menu is displayed, Select “/etc/microcode”.
6) A dialogue box may be displayed on screen. It will state that the current microcode level on the adapter is not in the /etc/microcode directory. This is acceptable because the adapter will reject any incorrect code. Press "Enter" to continue.
7) Select 000200091316 level and press "Enter" to flash the adapter.
8) The following message will appear on the screen when download is completed: "Microcode download complete successfully. The current microcode level for the device iba# is ... Please run diagnostics on the adapter to ensure that it is functioning properly."
9) If you selected more than one adapter to update, then steps 6-9 will repeat until all adapters are updated.
10) Exit diagnostics.

11.3 Verify and Re-configure adapters

1) Verify the code level is 000200091316 by typing "lsmcode –cd iba#" for each InfiniBand adapter updated. <Where # is the instance of the InfiniBand adapter>

2) Re-configure/Verify the adapter’s information (e.g. interface ip) through the “smit” menus.