IBM Microcode Update Readme File   PCIe2 2-port 10GbE SR Adapter (FC 5287) PCIe2 2-port 10GbE SR Low profile Adapter (FC 5284 ) PCIe2 2-port 10GbE SFP+Copper Adapter (FC 5288) PCIe2 2-port 10GbE SFP+Copper Low profile Adapter (FC 5286 )   ******* PLEASE READ THIS ENTIRE NOTICE *********  DATE: March 27, 2017 Table of Contents 1.0 Microcode and Document Revision History: 1 2.0 General information. 2 3.0 Installation Time. 2 4.0 Machine's Affected. 2 5.0 OS Requirements: 2 5.1 AIX.. 2 5.2 LINUX.. 2 6.0 Determine the Current Microcode Level 3 6.1 AIX.. 3 6.1 LINUX.. 3 7.0 Downloading the RPM Format File to the Target Server 4 7.1 AIX.. 4 7.2 LINUX.. 5 8.0 Discovery Tool Microcode CD-ROM creation and download instructions. 5 9.0 Verifying microcode level before download. 6 9.1 AIX.. 6 9.2 LINUX.. 6 10.0 Microcode Download Procedure: 6 10.1 AIX.. 6 10.1.1 Setting up for Microcode download. 6 10.2 Downloading Microcode to the Adapter 7 10.3 Verify and Re-configure adapters. 8 10.2 LINUX.. 8 10.2.1 Setting up for Microcode download. 8 10.2.2 Downloading Microcode to the Adapter 9 10.2.3 Verify and Re-configure adapters. 9 ===================================================================================== 1.0 Microcode and Document Revision History:   Firmware Level Description 0310204100102 Original Release 0310303970033 Adds network boot support and resets the base firmware level. October, 2011 0400003260003 Fixes EEH problem seen with Linux FCode fix for one port (port 1) not being configured (DEFINED), occasionally, especially during HOT PLUG Adds enforcement for supported cable type (10G Fiber-SR or Copper-Twinax) 0400401800007 Fixed link stability issues 0400401800009 Impact: Usability  Serverity: ATT Scheduler implemented to handle ioctl commands as a priority to prevent mailbox timeouts. Advanced Mode regression fix for FC5287/5284   Document Revision History Description 1.0 Initial release to drop code level 0310204100102 1.1 Adding Linux Update instruction 2.0 Release to drop code level 0310303970033 2.1 Adding firmware level 0400003260003 2.2 Adding firmware level 0400401800007 2.3 Section 5.0 OS Requirements content change 2.4 Adding firmware level 0400401800009 2.5 Added information about cable type enforcement in 0400003260003 and newer microcode levels. Revised instructions that mentioned the need to powercycle on linux after microcode update, changed to reboot oflpar needed on linux after microcode update. Adding Impact/Severity to microcode revision history 2.0 General information This Readme file is intended to give directions on how to update the microcode found on the PCIe2 2-port 10GbE SR Adapter Device ID for FC 5287, 5284:   a21910071410d003. Device ID for FC 5288, 5286:   a21910071410d103.   Key Note #1: This adapter requires an adapter reset after applying new microcode. If running network traffic, this means executing an LPAR reboot immediately after updating the desired cards, as the adapter will not function properly until reset. Updating the adapter in AIX does not require a reboot if you follow the instructions to temporarilyunconfigure and bring the network down on the adapter ports.   Key Note #2: Any microcode level beyond 0310303970033 will enforce the adapter feature code cable type. Please be sure to use the supported10Gb fiber-optics + transceiver module for FC 5287, 5284, or 10GbTwinaxial (Copper) cabling for FC 5288, 5286.   3.0 Installation Time Approximately 20 minutes per adapter. 4.0 Machine's Affected This adapter is supported in Power Systems that support PCIe Gen2 slots. 5.0 OS Requirements:             5.1 AIX                         AIX 61L technology level 6100-06 SP3 and later                         AIX 710 technology level 7100-00 SP2 and later             5.2 LINUX                                RHEL 6.0 and later 6.0 Determine the Current Microcode Level 6.1 AIX Before you install the microcode, it is important to determine the microcode level of the Adapter installed in the target system. Use the following instructions to read the ROM level stored in the Adapter's VPD. 1)  List the Ethernet adapters installed in the system by typing:     lsdev -C | grep ent Note: All of the Ethernet adapters that are installed will be displayed. They will be listed asentX, where X is 0, 1, etc. 2) To check the current microcode level for the adapter or controller enter the following command    lscfg -vl entX Where ?x? is the instance of the adapter. The command will produce output similar to: # lscfg -vl ent4   ent4             U78AA.001.WIH003H-P1-C1-C2-T1  PCIe2 2-port 10GbE SR Adapter(a21910071410d003)         PCIe2 2-port 10GbE SR Adapter, NIC PF:         Part Number.................74Y2094         Serial Number...............YL1025211026         Feature Code/Marketing ID...5287         FRU Number..................74Y2094         EC Level....................D76887         Manufacture ID..............1037         Network Address.............0000C9F235E6         ROM Level.(alterable)....... 0400401800009         Hardware Location Code......U78AA.001.WIH003H-P1-C1-C2-T1   If the ROM Level is less than 04 004 0180 0009 you should update the microcode. 6.2 LINUX   1)  Use this series of Linux commands to determine which of any installed Network adapters are Integrated Multifunction Adapters:   Note: It is advisable to cut and paste these command strings into your system console, to avoid typographical errors. Note: All of the Ethernet adapters that are installed will be displayed. They will be listed asethX, where X is 0, 1, etc. for i in `ls /sys/class/net | grep eth` ; do echo -n "${i}: "; ethtool -i ${i} | grep bus-info; done   Output will display the association between ethX devices and PCI bus device numbers,nnnn:nn:nn.n   eth0: bus-info: eth1: bus-info: 0000:01:00.0 eth2: bus-info: 0000:01:00.1 lspci | grep be3   Output will list PCI bus device numbers for the Integrated Multifunction Adapter functions,nnnn:nn:nn.n 0000:01:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) 0000:01:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) This example shows the two ports of one adapter. Firmware need only be updated on the first port of each adapter, eth1 in this case. If more than 2 ports are shown, it is likely you have more than one adapter, so you would need to also update firmware on first port of each adapter.   2) Use ethtool ?i ethX to find out which Ethernet interface is associated with BE3 adapter. If driver name is ?be2net? and the bus-info matches data from thelspci command, that interface is for Integrated Multifunction Card. For example:    #ethtool ?i ethX   Where ?x? is the instance of the adapter. The command will produce output similar to:   #ethtool ?i eth1   driver: be2net version: 4.1.307r firmware version: 4.4.180.9 bus-info: 0001:01:00.4   If driver name is ?be2net? and the bus-info matches the data from lspci, eth1 is one of Integrated Multifunction Card interfaces.  Firmware-version is ?4.4. 180.9?.    If firmware-version is less than 4.4.180.9 you should update the microcode. 7.0 Downloading the RPM Format File to the Target Server 7.1 AIX Use this method to download to an AIX system: NOTE: The instructions that follow are specific AIX commands. AIX commands are CASE (lower and upper) SENSITIVE, andmust be entered exactly as shown, including filenames. 1) Transfer the RPM format file to the /tmp directory (using ?Save as....?).     2) Install the microcode on your system using the rpm command as follows: # rpm -ihv --ignoreos /tmp/a21910071410d103.0400401800009.aix.rpm                 3) For AIX:  The microcode files will be added to /etc/microcode/.    NOTE:  - "/etc/microcode" is a symbolic link to "/usr/lib/microcode".  - If permission does not allow the copy to the above stated directory or file then the user will be prompted for a new location. 7.2 LINUX Use this method to download the new microcode to a Linux system: NOTE: The instructions that follow are specific Linux commands. Linux commands are CASE (lower and upper) SENSITIVE, andmust be entered exactly as shown, including filenames. 1)   Transfer the RPM format file to the /tmp directory (using ?Save as....?).   2)   Install the microcode on your system using the rpm command as follows: # rpm -ivh /tmp/a21910071410d103.0400401800009.Linux.rpm 3)    The microcode file will be added to /lib/firmware directory. Files name are: a21910071410d103.0400401800009 and a21910071410d003.0400401800009. File contents are identical and any can be used for firmware update on Linux.   8.0 Discovery Tool Microcode CD-ROM creation and download instructions Microcode Discovery Service is used to determine if microcode installed on yourIBM System P systems is at the latest level. To obtain information how to burn a CD-ROM and run the Discovery Tool for an AIX or Linux System please go to:http://www14.software.ibm.com/webapp/set2/mds/ 1) After running the Discovery Tool successfully the "/tmp/microcode/RPM" directory was created and your rpm files are copied from the CD-ROM. 2) Change to that directory, "cd /tmp/microcode/RPM". 3) Unpack the file by executing the instructions below:     Enter the commands:       rpm -ihv --ignoreos a21910071410d103.0400401800009.aix.rpm 4) One microcode file will be copied to "/etc/microcode".  The file size and checksum of the microcode image will be verified in Section 9.0.    File Name:             a21910071410d103.0400401800009 and a21910071410d003.0400401800009 NOTE:  - "/etc/microcode" is a symbolic link to "/usr/lib/microcode".  - If permission does not allow the copy to the above stated directory or file then the user will be prompted for a new location. 9.0 Verifying microcode level before download 9.1 AIX Please verify the file size and checksum of the raw microcode files matches what is listed below. For AIX : "ls -l /etc/microcode/a21910071410d003.0400401800009" to verify file size is 16777392. "sum /etc/microcode/a21910071410d003.0400401800009" to verify Checksum is 58478. "ls -l /etc/microcode/a21910071410d103.0400401800009" to verify file size is 16777392. "sum /etc/microcode/a21910071410d103.0400401800009" to verify Checksum is 58478. 9.2 LINUX "ls -l /lib/firmware/a21910071410d003.0400401800009" to verify file size is 16777392. "sum /lib/firmware/a21910071410d003.0400401800009" to verify Checksum is 58478 . "ls -l /lib/firmware/a21910071410d103.0400401800009" to verify file size is 16777392. "sum /lib/firmware/a21910071410d103.0400401800009" to verify Checksum is 58478 .   10.0 Microcode Download Procedure:  Microcode download must be performed separately for each Adapter under Partitions (LPAR's) that owns the adapter.             10.1 AIX 10.1.1 Setting up for Microcode download 1.     Stop all applications that use this interface/adapter 2.     Remove the interface/IP address from the all ports identified in section 6.0 for the adapters that will be upgraded. a.     Before detaching the interface, record the ip address and any other pertinent information that was configured on the Adapter.  This information may be needed if the microcode update overwrites this section on the Adapter. 3.     If the interfaces are members of a Shared Ethernet Aadapter (SEA), the SEA devices must be moved to a defined state. a.     ?rmdev -l enX? ? where ?X? is the interface number for the Shared Ethernet Adapter. b.     ?rmdev ?l entX? ? where ?X? is the interface number for the Shared Ethernet Adapter. 4.     If the interfaces are members of an EtherChannel, the EtherChannel device must be moved to a define state. a.     ?rmdev -l enX? ? where ?X? is the interface number for the EtherChannel adapter. b.     ?rmdev ?l entX? ? where ?X? is the interface number for the EtherChannel adapter. 5.     For every port associated with the adapter, the enX interfaces must be changed to a defined state. a.     ?rmdev -l enX? - where ?X? is the interface number for the adapter port b.     This command will be run 2 times for each adapter, once for each port on the adapter.     10.2 Downloading Microcode to the Adapter 1) At the command line type ?diag? 2) Select the "Task Selection" from diagnostics menu. 3) Select "Microcode Tasks? then select ?Download Microcode" from the menu. 4) Select one entX device for each of the 2-port 10GbE SR Adapter(a21910071410d003)that need to be updated from the list of devices by using the arrow keys to highlight the entry and pressing "Enter" to mark it.  The devices will be displayed asent#, where # is the number of the device.  Press "F7" when you are done mark all the adapters you want to flash. 5) Select ?/etc/microcode? or "/usr/lib/microcode". 6) A dialogue box may be displayed on screen.  It will state that the current microcode level on the adapter is not in the /etc/microcode directory.  This is acceptable because the adapter will reject any incorrect code.  Press "Enter" to continue. 7) Select 0400401800009 level and press "Enter" to flash the adapter. 8) The following message will appear on the screen when download is completed: "Microcode download complete successfully.  The current microcode level for the deviceent# is ...  9) If you selected more than one adapter to update, then steps 6-9 will repeat until all adapters are updated. 10) Exit diagnostics. 10.3 Verify and Re-configure adapters 1) Verify the code level is 0400401800009 by typing "lscfg -vl ent#" for each Ethernet adapter updated (where # is the instance of the Ethernet adapter). 2) Re-configure/Verify the adapter?s interface ip information through the ?smit? menus.             10.2 LINUX    10.2.1 Setting up for Microcode download   1)  Use this series of Linux commands to determine which of any installed Network adapters areIntegrated Multifunction Adapters:       Note: It is advisable to cut and paste these command strings into your system console, to avoid typographical errors. for i in `ls /sys/class/net | grep eth` ; do echo -n "${i}: "; ethtool -i ${i} |grep bus-info; done Output will display the association between ethX devices and PCI bus device numbers,nnnn:nn:nn.n      eth0: bus-info: eth1: bus-info: 0000:01:00.0 eth2: bus-info: 0000:01:00.1 lspci | grep be3      Output will list PCI bus device numbers for the Integrated Multifunction Adapter functions,nnnn:nn:nn.n 0000:01:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) 0000:01:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) This example shows the two ports of one adapter. Firmware need only be updated on the first port of each adapter, eth1 in this case. If more than 2 ports are shown, it is likely you have more than one adapter, so you would need to also update firmware on first port of each adapter. 2)   Stop all applications that use the interface/adapter to be updated 3)   Configure the adapter?s interface, eth1 in this example. ifconfig eth1 192.168.1.1/24 4)   Check that no errors were reported. If errors were reported verify proper interface was configured and command was entered correctly.    10.2.2 Downloading Microcode to the Adapter Please be sure to reset the adapter after updating microcode. See General Information Key Note #1 in Section 2.0.   1)   Using the ethX configured in previous step, use ethtool ?flash to update the firmware.  Note: RPM installed three images; the images are identical in content and only different by name.              ethtool -f ethX a21910071410d003.0400401800009 2)   If flashing is completed successfully, no messages will appear on the screen. You can check messages in system log.              dmesg | grep be2net      For example, dmesg command will show the following messages:      be2net 0000:01:00.0: firmware: requesting a21910071410d003.0400401800009 be2net 0000:01:00.0: Flashing firmware file a21910071410d003.0400401800009 be2net 0000:01:00.0: Firmware flashed successfully These messages show flashing is completed successfully. 3)  Verify firmware has been activated Using same ethX  interface used for ethtool ?flash command, type: ethtool -i ethX driver: be2net version: 4.1.307r firmware-version: 4.4.180.7 [4.4.180.9] bus-info: 0000:01:00.0 4)   Firmware-version must show 4.4.180.9.  If firmware-version is not as expected, carefully retrace steps to confirm no steps were omitted or performed incorrectly.If no errors or omissions were found, contact IBM customer service for assistance. 5)  On a non hot-pluggable system/slot where you install adapter, you have to reboot the entire system (not just LPAR reboot or shutdown) for the update to take effect. On hot-pluggable system/slot, LPAR reboot is sufficient for the new FW to load.    10.2.3 Verify and Re-configure adapters 1) Verify the code level is 4.004.180.009 by typing "ethtool ?i ethX" for each Ethernet adapter updated. 2) Re-configure/Verify the adapter?s interface ip by ?ifconfig ethX? . Be aware that this adapter requires a slot reset (See Key Note #1 in Section 2.0). It will not function properly until this is performed.