IBM Microcode Update Readme File          (#EN10) - Integrated Mutifunction 
Card with 10GbE RJ45 & Copper Twinax                             

(#EN11) - Integrated Multifunction Card with 10GbE RJ45 & SR Optical

******* PLEASE READ THIS ENTIRE NOTICE *********


 DATE: April 19, 2018 Table of Contents 

1.0       Microcode and Document Revision History: 1

2.0 General information  4

4.0 Machine's Affected: 4

5.0 OS Requirements: 4

6.1 AIX   5

7.0 Downloading the RPM Format File to the Target Server  9

7.1 AIX   10

7.2 LINUX   10

8.0 Discovery Tool Microcode CD-ROM creation and download instructions  11

9.0 Verifying microcode before download and notes  12

9.1 AIX   12

9.2 LINUX   12

10.0 Microcode Download Procedure: 12

10.1 AIX   12

10.1.1 Setting up for Microcode download  12

10.1.2 Downloading Microcode to the Adapter 13

10.1.3 Verify and Re-configure adapters  13

10.2 LINUX   13

10.2.1     Setting up for Microcode download  13

10.2.2 Downloading Microcode to the Adapter 14


=====================================================================================

1.0           Microcode and Document Revision History: 
   

Microcode Level

Changes

 

 

00011000020021100021

 

 

 

Impact: Usability   Severity: ATT

Fix for the following:

* adapter has an intermittant issue where reading NIC fcode returns 00

* FCode memory leak in fibre channel driver

* adapter with FW 11.1.35.67 failing NIM install

* Direct Attach LinkUp Issue with DS8K array. FEC set to off on both sides to 
work.

* Add data scrubbing to adapter FW dump process to remove/obscure user data

* Customer reporting spontaneous dump on FCoE adapter running 10.2.252.1921 fw

* FC adapter intermittently aborts active I/Os after Accepted REC

* VIOS crashed when lpar was booted into Sles

The Firmware Levels Below Are No Longer Supported By IBM Once They Have Been 
Removed From The Microcode Down Load Website.

It is best practices to update to the latest FW level not only for IBM 
support of these products, but for optimal performance and to ensure that all 
of the required HW/FW fixes are installed. Please Update To The Latest Level At 
Your Earliest Convenience

00010000020025201905

*Fix for SRIOV mode, adapter went to error state when config Vnic. 

*Fix for SRIOV VF TX timeouts when other VFs are FLR ? reseted.

*Fix for SRIOV  Vnic ping failure when injecting EEH errors.

*Fix for spontaneous livedump with adapter entered error state 
(9F000013,00001007)

*Fix a rogue frame with EoFa and a bad length, seen during switch reboots.

*Fix an inbound class 2 FLOGI rejected with non-retryable invalid class of 
service

00010000020025200016

*Fix for SRIOV race conditions in PCI Function Reset

*Fix for Outbound Jammer test (Dropped outbound data frame of single frame 
sequence)

*Fix IO fail to resume after f/w update on FCoE

*Fix DMA to 0 in an IBMi system

*Fix for SRIOV hard LPAR shutdowns

00010000020025200009

* FC Adapter Port Error on ISL pull 

* P_RJT response for Fibre Channel Class 2 ELS's 

* Reboot FCoE adapter path recovery failure - mailbox error and fcs unknown 
error

* FC adapter does not indicate closed exchange when WQE with LNK bit set 
fails (fw ignores early closure of exchange  for linked scsi commands) 

* Experiencing NIM install issues with adapters                             

* DCBX Get Config Mailbox Cmd timing out. Fix Link bounces   

* Unable to set pause Flow control when connected to DCBX switch with PFC off

* GRP5 Link COS ACQE needs to be delivered on DCBX/PFC change

* AIX issues BA_RJT with reason code "Protocol Error" to target-Initiated ABTS

     00001000010005800004 

 

- Exchanges aborted after REC is successfully ACC'd by target device.

- COMMON_GET_FLOW_CONTROL issues in dedicated mode

- No link after cable plugged in

- VIOS disk path does not recover from Initiator port cable pull

- Adapter pport stat counter TX_BROADCAST_BYTES incorrect

- Logging in and out of fabric 

 

00001000010003400005

1) Adapter livedump following CRC forced errors.

2) fcode fix of SAN boot LPAR stuck at LED  AA00E158

3) No link after cable plugged in.

4) Failed paths to SAN after a switch reboot.

 

00001000010003400003

This is the level of microcode that initially shipped with the adapter when 
it became generally available.

 

 

Document Revision History

Description

1.0

 Initial release to drop code level  00001000010003400003

5/28/2013

Update readme file with level  00001000010003400005

6/23/2014

Update readme file with level  00001000010005800004

12/3/2014

Update readme file with level  00010000020025200009 

6/15/2015

Update readme file with level  00010000020025200016 

2/17/2016

Update readme file with level  00010000020025201905

4/19/2018

Update readme file with level  00011000020021100021

 

 


2.0 General information 

This Readme file is intended to give directions on how to update the 
microcode found on the Integrated Multifunction Card with 10GbE RJ45 & Copper 
Twinax(FC EN10, CCIN 2C4C) and the Integrated Multifunction Card with 10GbE 
RJ45 & SR Optical(FC EN11, CCIN 2C4D)

Device ID for FC EN10, CCIN 2C4C: df1060e214103704

Device ID for FC EN11, CCIN 2C4D: df1060e214103a04

Cautions and Important Notes


 * Concurrent Download The microcode installation supports concurrent download 
while the fcs resources and attached storage devices are available for use. 
HOWEVER, concurrent download is NOT supported for the ent resources. If 
the entresources are in use, they must manually be released prior to starting 
the download. Failure to do so will result in a message instructing that 
the ent resource be freed. 

NOTE: It is HIGHLY recommended that the installation be scheduled during a 
maintenance window or during non-peak production periods.

 

 

 

3.0 Installation time

Approximately 20 minutes per adapter.

4.0 Machine's Affected: 

This adapter is supported in Power Systems 9117-MMC and 9179-MHC.

5.0 OS Requirements:

Basic card function is supported on:

            Red Hat Enterprise Linux 6.4 for POWER® , and later 

            SUSE Linux Enterprise Server 11 Service Pack 2, and later, with 
current maintenance updates available from SUSE to enable all planned 
functionality



Download drivers link:

http://www.emulex.com/downloads/emulex/drivers/linux/

 

            AIX 61N technology level 6100-02 and later

            AIX 71D technology level 7100-03 and later

 

          

Important Notice:  EEH error entries in AIX system errlog are expected when 
updating adapter FW.

 

 

 

6.0 Determine the Current Microcode Level: 

  6.1 AIX

Before you install the microcode, it is important to determine the microcode 
level of the Adapter installed in the target system. Use the following 
instructions to read the ROM level stored in the Adapter's VPD.

1)  List the Ethernet or Fiber Channel adapters installed in the system by 
typing:

lsdev -C | grep ent 

lsdev -C | grep fcs

Note: All of the adapters that are installed will be displayed. They will be 
listed as entX or fcsX, where X is 0, 1, etc. 

2) To check the current microcode level for the adapter or controller enter 
the following command

   lscfg -vl entX

   lscfg -vl fcsX

Where ?x? is the instance of the adapter. The command will produce output 
similar to:

EN10

 

> lscfg -vl ent2

  ent2             U2C4E.001.DBJ3257-P2-C8-T1  10GBaseT 4-port Integrated 
Multifunction CNA (df1020e214103b04)

 

      Ethernet Adapter:

        Network Address.............5CF3FCCF02F4

        ROM Level.(alterable).......11.2.211.21

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8-T1

 

> lscfg -vl ent5

  ent5             U2C4E.001.DBJ3257-P2-C8-T4  10GbE SFP+ Cu 4-port 
Integrated Multifunction CNA (df1020e214103904)

 

      Ethernet Adapter:

        Network Address.............5CF3FCCF02F7

        ROM Level.(alterable).......11.2.211.21

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8-T4

 

> lscfg -vl fcs2

  fcs2             U2C4E.001.DBJ3257-P2-C8-T3  Integrated 10Gb Cu 4-Port FCoE 
Adapter (df1060e214103a04)

 

        Part Number.................OCl15104-FM-P

        Serial Number...............00000000

        Network Address.............10005CF3FCCF02FA

        Device Specific.(Z0)........0000000B

        Device Specific.(Z1)........00000010

        Device Specific.(Z2)........00000000

        Device Specific.(Z3)........08090000

        Device Specific.(Z4)........00023201

        Device Specific.(Z5)........33342E33

        Device Specific.(Z6)........33342E33

        Device Specific.(Z7)........C0022B40

        Device Specific.(Z8)........20005CF3FCCF02FA

        Device Specific.(Z9)........11.2.211.21

        Device Specific.(ZA)........11.2.211.21

        Device Specific.(ZB)........30313233

        Device Specific.(ZC)........00040000

        Device Specific.(ZD)........000000FF

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8-T3

 

> lscfg -vpl sysplanar0 | grep -p ETH

      DUAL 1&10GBT ETH:

        Record Name.................VINI

        Flag Field..................XXET

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8

        Customer Card ID Number.....2C4C

        Serial Number...............YL30BG28P004

        CCIN Extender...............1

        Product Specific.(VZ).......01

        FRU Number..................00E2907

        Part Number.................00E2908

        Product Specific.(HE).......0001

        Product Specific.(CT).......30910008

        Product Specific.(HW).......0001

        Product Specific.(B3).......000000000001

        Product Specific.(B4).......00

        Product Specific.(B7).......000000000000000000000000

        Version.....................ipzSeries

      Physical Location: U2C4E.001.DBJ3257-P2-C8

 


------------------------------------------------------------------------------------------------------------

EN11

 

> lscfg -vl ent2 

  ent2             U2C4E.001.DBJ3257-P2-C8-T1  10GBaseT 4-port Integrated 
Multifunction CNA (df1020e214103804)

 

      Ethernet Adapter:

        Network Address.............5CF3FCCF5607

        ROM Level.(alterable).......11.2.211.21

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8-T1

 

> lscfg -vl ent4

  ent4             U2C4E.001.DBJ3257-P2-C8-T3  10GbE SFP+ SR 4-port 
Integrated Multifunction CNA (df1020e214103604)

 

      Ethernet Adapter:

        Network Address.............5CF3FCCF5609

        ROM Level.(alterable).......11.2.211.21

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8-T3

 

> lscfg -vl fcs0

  fcs0             U2C4E.001.DBJ3257-P2-C8-T3  Integrated 10Gb 4-Port FCoE 
Adapter (df1060e214103704)

 

        Part Number.................OCl15104-FM-P

        Serial Number...............00000000

        Network Address.............10005CF3FCCF560D

        Device Specific.(Z0)........0000000B

        Device Specific.(Z1)........00000010

        Device Specific.(Z2)........00000000

        Device Specific.(Z3)........08090000

        Device Specific.(Z4)........00023201

        Device Specific.(Z5)........33342E33

        Device Specific.(Z6)........33342E33

        Device Specific.(Z7)........C0022B40

        Device Specific.(Z8)........20005CF3FCCF560D

        Device Specific.(Z9)........11.2.211.21

        Device Specific.(ZA)........11.2.211.21

        Device Specific.(ZB)........30313233

        Device Specific.(ZC)........00040000

        Device Specific.(ZD)........000000FF

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8-T3

 

> lscfg -vpl sysplanar0 | grep -p ETH  

      DUAL 1&10GBO ETH:

        Record Name.................VINI

        Flag Field..................XXET

        Hardware Location Code......U2C4E.001.DBJ3257-P2-C8

        Customer Card ID Number.....2C4D

        Serial Number...............YL10JH2CZ036

        CCIN Extender...............1

        Product Specific.(VZ).......01

        FRU Number..................00E2913

        Part Number.................00E2914

        Product Specific.(HE).......0001

        Product Specific.(CT).......30910008

        Product Specific.(HW).......0001

        Product Specific.(B3).......000000000001

        Product Specific.(B4).......00

        Product Specific.(B7).......000000000000000000000000

        Version.....................ipzSeries

      Physical Location: U2C4E.001.DBJ3257-P2-C8

            If the ROM Level is less than 11.2.211.21 you should update the 
microcode.

6.2 Linux

NOTE: The instructions that follow are specific Linux commands. Linux 
commands are CASE (lower and upper) SENSITIVE, andmust be entered exactly as 
shown, including filenames.

1)  List the adapter bus-info for Integrated Multifunction adapters installed 
in the system by typing:

    lspci | grep Lancer

    [root@io91 ~]# lspci | grep Lancer

     0001:01:00.0 Ethernet controller: Emulex Corporation OneConnect NIC 
(Lancer) (rev 10)
     0001:01:00.1 Ethernet controller: Emulex Corporation OneConnect NIC 
(Lancer) (rev 10)
     0001:01:00.2 Ethernet controller: Emulex Corporation OneConnect NIC 
(Lancer) (rev 10)
     0001:01:00.3 Ethernet controller: Emulex Corporation OneConnect NIC 
(Lancer) (rev 10)
     0001:01:00.6 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator 
(Lancer) (rev 10)
     0001:01:00.7 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator 
(Lancer) (rev 10)

2)  List the Ethernet interfaces for Integrated Multifunction adapters 
installed in the system by typing:

    ifconfig -a | grep eth

Note: All of the Ethernet adapters that are installed will be displayed. They 
will be listed as eth{n}, where {n} is 0, 1, etc.

2) For each adapter listed, use ethtool -i eth{n} to determine which Ethernet 
interfaces are associated with the Integrated Multifunction Card. If driver 
name is ?be2net? and the bus-info matches data from the lspci command, that 
interface is for Integrated Multifunction Card. For example:

   ethtool -i eth{n}

Where {n} is the instance of the adapter. The command will produce output 
similar to:

ethtool -i eth1

driver: be2net
 version: 4.1.307r
 firmware version: 11.2.211.21
bus-info: 0001:01:00.0

If driver name is ?be2net? and the bus-info matches the data from lspci, eth1 
is one of Integrated Multifunction Card interfaces. Firmware-version is ?
11.2.211.21?.

          If firmware-version is below 11.2.211.21  on any of the adapters 
listed with a be2net driver; you should update the microcode.



7.0 Downloading the RPM Format File to the Target Server

7.1 AIX

Use this method to download to an AIX system: 

NOTE: The instructions that follow are specific AIX commands. AIX commands 
are CASE (lower and upper) SENSITIVE, andmust be entered exactly as shown, 
including filenames.

1) Make two directories on your AIX system to receive the RPM format file. 
     Enter:      "mkdir /tmp/microcode" 

   and then create this directory

    Enter:      "mkdir /tmp/microcode/RPM" 

2) Transfer the RPM format file to the /tmp/microcode/RPM directory (using 
"Save as ...").  Change to that directory, "cd /tmp/microcode/RPM". 

               You'll see the filename for the RPM file.


             "rpm -ivh --ignoreos df1060e214103.00011000020021100021
.aix.rpm"                                      

 

3) For AIX:  The microcode files will be added to /etc/microcode/.  

 

NOTE: 
  - "/etc/microcode" is a symbolic link to "/usr/lib/microcode". 
  - If permission does not allow the copy to the above stated directory or 
file then the user will be prompted for a new location.

7.2 LINUX

Use this method to download the new microcode to a Linux system: 

NOTE: The instructions that follow are specific Linux commands. Linux 
commands are CASE (lower and upper) SENSITIVE, andmust be entered exactly as 
shown, including filenames.

1)   Transfer the RPM format file to the /tmp directory(using ?Save as....?). 
You will see the filename for theRPM file.  

2)   Install rpm on your Linux system by typing:

                    rpm -ivh /tmp/df1060e214103.00011000020021100021.Linux.rpm

3) The microcode package will install the firmware images in the 
/lib/firmware directory. If a message is displayed saying the "package 
<package_name> is already installed", you will need to uninstall the listed rpm 
package. On the command line type:

rpm -e <package_name>

where <package_name> is the name of the package that was returned in the 
message. Return to Step 2 and attempt to install the file again.

          4)    The microcode files will be installed to the /lib/firmware 
directory. File names are? df1060e214103704.00011000020021100021 and  
df1060e214103a04.00011000020021100021

  File contents are identical, either can be used for firmware update on 
Linux.

8.0 Discovery Tool Microcode CD-ROM creation and download instructions 

To obtain information how to burn a CD-ROM and run the Discovery Tool for an 
AIX or Linux System please go to:http://www14.software.ibm.com/webapp/set2/mds/

1) After running the Discovery Tool successfully the "/tmp/microcode/RPM" 
directory was created and your rpm files are copied from the CD-ROM.

2) Change to that directory, "cd /tmp/microcode/RPM". 

             3) Unpack the file by executing the instructions below: 
                   Enter the command: 
                       rpm -ihv --ignoreos df1060e214103.00011000020021100021.a
ix.rpm

4) Two microcode files will be copied to "/etc/microcode".  The file size and 
checksum of the microcode image will be verified in Section 9.0.
    File Name:


                           /etc/microcode/df1060e214103a04.00011000020021100021

 

                           /etc/microcode/df1060e214103704.
00011000020021100021

 

 

NOTE: 
  - "/etc/microcode" is a symbolic link to "/usr/lib/microcode". 
  - If permission does not allow the copy to the above stated directory or 
file then the user will be prompted for a new location.

9.0 Verifying microcode before download and notes

9.1 AIX

Please verify the file size and checksum of the raw microcode files matches 
what is listed below.

                        ls -l /etc/microcode/df1060e214103*.
00011000020021100021

                              to verify file size is 10846192.
                       sum /etc/microcode/df1060e214103*.00011000020021100021

                              to verify Checksum is 43623.

9.2 LINUX

Please verify the file size and checksum of the raw microcode files matches 
what is listed below.

                     ls -l /lib/firmware/df1060e214103*.00011000020021100021

                          to verify file size is 10846192 for both files 
listed


                     sum /lib/firmware/df1060e214103*.00011000020021100021

                 to verify Checksum is 43623 for both files listed

10.0 Microcode Download Procedure:  

            10.1 AIX

            10.1.1 Setting up for Microcode download

                        Detach the Adapter?s interface (not always needed, 
but it is a good practice to do this step).

Before detaching the interface, record the ip address and any other pertinent 
information that was configured on the Adapter.  This information may be needed 
if the microcode update overwrites this section on the Adapter.

Note: If NIC ports are used in SEA Shared Ethernet Adapter, or EtherChannel. 
Detach and rmdev are needed:

  ifconfig en# detach

  rmdev -Rl ent# 

 

 

 

                       You only need to pick one fcs# per adapter for 
microcode update. Diag will update microcode for all ports. Note:  you might 
need to perform ?rmdev -Rl fscsi#  ? to remove child devices to the fcs#. 

                        fcs# needs to be Available for fw download

 

            10.1.2 Downloading Microcode to the Adapter

 

                           1) At the command line type "diag"
                           2) Select the "Task Selection" from diagnostics 
menu.
                           3) Select "Microcode Tasks" then select "Download 
Microcode" from the menu.
                           4) Select one fcs# or ent# for each of the Int 
Multifunction Cardsthat need to be updated from the list of devices by using 
the arrow keys to highlight the entry and pressing "Enter" to mark it. 

                                 Press "F7" when you are done marking all the 
adapters you want to flash.
                           5) Select "/etc/microcode" or "/usr/lib/microcode".
                           6) A dialogue box may be displayed on screen.  It 
will state that the current microcode level on the adapter is not in the 
/etc/microcode directory. 

                               This is acceptable because the adapter will 
reject any incorrect code.  Press "Enter" to continue.
                           7) Select  00011000020021100021 level and press 
"Enter" to flash the adapter.
                           8) The following message will appear on the screen 
when download is completed: "Installation of the microcode has completed 
successfully ...  "
                           9) If you selected more than one adapter to update, 
then steps 6-9 will repeat until all adapters are updated.
                          10) Exit diagnostics.

            10.1.3 Verify and Re-configure adapters

                           

                           Verify the code level is  00011000020021100021

 by typing "lscfg -vl ent#" or "lscfg -vl fcs#" for each adapter updated. 
(where # is the instance of the Ethernet adapter).
 

            10.2 LINUX

10.2.1  Setting up for Microcode download

1)    Use this series of Linux commands to determine which of any installed 
Network adapters areIntegrated Multifunction Adapters:
 Note: It is advisable to cut and paste these command strings into your system 
console, to avoid typographical errors.

for i in `ls /sys/class/net | grep eth` ; do echo -n "${i}: "; ethtool -i ${i} 
| grep bus-info; done

 Output will display the association between eth{n} devices and PCI bus device 
numbers, nnnn:nn:nn.n

 eth0: bus-info:
 eth1: bus-info: 0001:01:00.0
 eth2: bus-info: 0001:01:00.1
 eth3: bus-info: 0001:01:00.2
 eth4: bus-info: 0001:01:00.3

lspci -d 10df:e220

 Output will list PCI bus device numbers for the Integrated Multifunction 
Adapter functions, nn:nn.n

 0001:01:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) 
(rev 10)
 0001:01:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) 
(rev 10)
 0001:01:00.2 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) 
(rev 10)
 0001:01:00.3 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer) 
(rev 10)

 This example shows the four ports of one Integrated Multifunction Adapter. 
Firmware need only be updated on the first port of each adapter,
 eth1 in this case.

 If more than 4 ports are shown, it is likely you have more than one adapter, 
so you would need to also update firmware on first port of second
 adapter.



2)    Stop all applications that use the interface/adapter to be updated

3)    Configure the adapter?s interface, eth1 in this example.

ifconfig eth1 192.168.1.1/24

4)    Check that no errors were reported. If errors were reported verify 
proper interface was configured and command was entered correctly.

   10.2.2 Downloading Microcode to the Adapter

1)    Using the eth{n} configured in previous step use ethtool --flash to 
update the firmware.
 Note that RPM installed two images; the images are identical in content and 
only different by name.

                                               ethtool --flash  eth{n} 
df1060e214103a04.00011000020021100021

 

       Note: eth{n} is the interface previously configured with ifconfig  
(See 10.2.1 Setting up for Microcode downloading).

2)    If flashing is completed successfully, no messages will appear on the 
screen. You can check messages in system log.

dmesg | grep be2net

For example, dmesg command will show the following messages:

                                     be2net 0001:01:00.1: firmware: requesting
  df1060e214103a04.00011000020021100021
                                     be2net 0001:01:00.1: Flashing firmware 
file df1060e214103a04.00011000020021100021


 be2net 0001:01:00.1: Firmware flashed successfully 

 These messages show flashing is completed successfully.

3)    Verify firmware has been activated

 Using same eth{n} interface used for ethtool --flash command, type:

 ethtool -i eth1

 driver: be2net
 version: 4.1.307r
 firmware-version: 11.2.211.21
 bus-info: 0001:01:00.1



4)    Firmware-version must show 11.2.211.21.  If firmware-version is not as 
expected, carefully retrace steps to confirm no steps were omitted or performed 
incorrectly. If no errors or omissions were found, contact IBM customer service 
for assistance.