PCIe2
10Gb 4-Port FCoE Adapter
*******
PLEASE READ THIS ENTIRE NOTICE *********
DATE: October 20, 2020
Table of Contents
Contents
1.0 Microcode and Document
Revision History:1
7.0 Determine the Current
Microcode Level for AIX:5
8.0
Downloading the RPM Format File to the Target Server
8.1
AIX Use
this method to download to an AIX system:
10.0 Verifying microcode
before download and notes8
11.0 Microcode Download
Procedure for AIX:9
11.1 Setting up for Microcode download 9
11.2 Downloading Microcode to the Adapter 9
11.3 Verify and Re-configure adapters 10
12.0 Microcode Download
Procedure for Linux:11
12.1 Setting up for Microcode download 11
12.2 Downloading Microcode to the Adapter 12
|
Microcode Level |
Changes |
|
00011000040041500033 |
Impact: Data
Severity: HIPER Fix for the following: * Fix for a condition
that may result from frequent resets of adapter Virtual Functions or
transmission stalls and could lead to potential undetected data corruption |
|
The Firmware Levels Below
Are No Longer Supported By IBM Once They Have Been Removed From The Microcode
Down Load Website. It is best practices to
update to the latest FW level not only for IBM support of these products, but
for optimal performance and to ensure that all of the required HW/FW fixes
are installed. Please Update To The Latest Level At Your Earliest Convenience |
|
|
00011000040041500016 |
Impact: Usability Severity: ATT Fix for the following: *Scanning adapter in SMS when direct attached
to V7000 takes over 2 minutes-fcode *a fatal error state after some Admin WQE
failures *Adapter EEH error because status frame sent
incorrectly. *
Change mailbox tracking timeout to 25 secs for fast commands doing flash
operations *
Release FFS read lock during non critical sections of garbage collection *
Copyright information showing incorrect for Fcode *
This level of firmware supports the Firmware Secure Boot feature of Power
systems *
For open and unused ports experiencing LNC2ENT_HW_TMP_ERRs, this fix is
partial and requires IBM APAR IJ10293 for full correction. |
|
00011000020021100021 |
* adapter has an intermittant issue
where reading NIC fcode returns 00 * FCode memory leak in fibre channel
driver * adapter with FW 11.1.35.67 failing
NIM install * Direct Attach LinkUp Issue with DS8K
array. FEC set to off on both sides to work. * Add data scrubbing to adapter FW
dump process to remove/obscure user data * Customer reporting spontaneous dump
on FCoE adapter running 10.2.252.1921 fw * FC adapter intermittently aborts
active I/Os after Accepted REC * VIOS crashed when lpar was booted into Sles |
|
00010000020025201927 |
Fix for the following: * TX
stall causing back-pressure in the ASIC resulting in mailbox response not being sent to host. * System Crash when LPAR was booted into SLES * Hardware Ethernet Error caused by a dead
lock in adapter FW. A mutex was taken and not released by mistake. *
Adapter
enters firmware loop when single bit ECC error is detected. System * Fix direct-attach issue with DS8000 storage |
|
00010000020025201913 |
Fixed an
intermittent issue Link not coming up after reboot with Cisco switch. |
|
00010000020025201905 |
*Fix for SRIOV mode, adapter went to error
state when config Vnic. *Fix for SRIOV VF TX timeouts when other VFs
are FLR – reseted. *Fix for SRIOV
Vnic ping failure when injecting EEH errors. *Fix for spontaneous livedump with adapter
entered error state (9F000013,00001007) *Fix a rogue frame with EoFa and a bad length,
seen during switch reboots. *Fix an inbound class 2 FLOGI rejected with
non-retryable invalid class of service |
|
00010000020025200016 |
*Fix for SRIOV race conditions in PCI Function
Reset *Fix for Outbound Jammer test (Dropped
outbound data frame of single frame sequence) *Fix
IO
fail to resume after f/w update on FCoE *Fix DMA to 0 in an IBMi system *Fix for SRIOV hard LPAR shutdowns |
|
00010000020025200009 |
* FC * P_RJT response for Fibre Channel Class 2
ELS's * Reboot FCoE adapter path recovery failure -
mailbox error and fcs unknown error * FC adapter does not indicate closed exchange
when WQE with LNK bit set fails (fw ignores early closure of exchange for linked scsi commands) * Experiencing NIM install issues with
adapters * DCBX Get Config Mailbox Cmd timing out. Fix
Link bounces * Unable to set pause Flow control when
connected to DCBX switch with PFC off * GRP5 Link COS ACQE needs to be delivered on
DCBX/PFC change * AIX issues BA_RJT with reason code
"Protocol Error" to target-Initiated ABTS |
|
00001000010005800004 |
- Exchanges aborted after REC is successfully ACC'd by
target device. -
COMMON_GET_FLOW_CONTROL issues in dedicated mode - No link
after cable plugged in - VIOS
disk path does not recover from Initiator port cable pull - Adapter
pport stat counter TX_BROADCAST_BYTES incorrect - Logging
in and out of fabric |
|
00001000010003400005 |
1)
Adapter livedump following CRC forced errors. 2) fcode
fix of SAN boot LPAR stuck at LED
AA00E158 3) No
link after cable plugged in. 4) Failed
paths to SAN after a switch reboot. |
|
00001000010003400003 |
This is the level of microcode that initially shipped
with the adapter when it became generally available. |
|
Document
Revision History |
Description |
|
|
Creating Readme file with
latest microcode for this adapter - df1060e214101004 |
|
|
Update readme file with level 00001000010003400005 |
|
06/06/2014 |
Update readme file with level
00001000010005800004 |
|
11/14/2014 |
Update to level 00010000020025200009 |
|
6/4/2015 |
Update to level 00010000020025200016 |
|
9/14/2015 |
Update to level 00010000020025201905 |
|
4/21/2016 |
Update to level 00010000020025201913 |
|
11/11/2016 |
Added Low Cost Feature Codes EL56 and EL38 |
|
4/28/2017 |
Update to level 00010000020025201927 |
|
1/29/2018 |
Update to level 00011000020021100021 |
|
4/4/2018 |
Added Caution Note Maint Windows |
|
12/05/2018 |
Update to level 00011000040041500016 |
|
10/18/2019 |
Updated section 11.3Verify and Re-configure
adapters with #lscfg -vl output examples |
|
10/20/2020 |
Update to level 00011000040041500033 |
This
Readme file is intended to give directions on how to update the microcode found
on the PCIe2 10Gb 4-Port FCoE Adapter (FC: EN0H , EN0J, EL56 and EL38 ).
Approximately 20 minutes.
Feature Codes EN0H ( Full Height ), EN0J ( Low Profile
)
Low Cost Feature Codes EL56 ( Full Height ), EL38 ( Low Profile )
CCIN 2B93
Cautions and Important Notes
·
Concurrent Download
The microcode installation supports concurrent download
while the adapter and attached storage devices are available for use. HOWEVER,
it is recommended that installation be scheduled during a maintenance window or
during non-peak production periods.
·
Red Hat Enterprise Linux 6.4 for POWER® , or later
·
SUSE Linux
Download drivers link:
http://www.emulex.com/downloads/emulex/drivers/linux/
Basic card
function is supported on :
If you are
using another release of AIX, ensure that the adapter is supported on that
release before you install the adapter. Contact service and support for
assistance.
Important
Notice: If your AIX system doesn’t have the APARs
listed below:
You need to unconfigure the ethernet ports before updating adapter FW to
avoid Ethernet errors.
This is only if you update from 1.1.34.5 (00001000010003400005) into
1.1.58.4 ( 00001000010005800004 )
|
AIX Level |
APAR
needed |
|
7100-01-06 and
later |
IV57610 |
|
7100-02-02 and
later |
IV57070 |
|
7100-03 and
later |
IV56321 |
|
|
|
|
6100-07-06 and
later |
IV58726 |
|
6100-08-02 and
later |
IV58005 |
|
6100-09 and
later |
IV56310 |
Important
Notice: EEH
error entries in AIX system errlog are expected when updating adapter FW.
Before you install the
microcode, it is important to determine the microcode level of the Adapter
installed in the target system. Use the following instructions to read the ROM
level stored in the Adapter's VPD.
1) List the Fiber
Channel adapters or Ethernet adapters installed in the system by
typing:
lscfg -l fcs\*
or:
lscfg -l ent\*
Note: All of the Fiber
Channel and Ethernet adapter ports that are installed will be displayed. They
will be listed as fcsX and entX, where X is 0, 1, etc.
2) To check the current
microcode level for the adapter or controller enter the following command
lsmcode -cd
fcsx
or:
lsmcode -cd entx
Where “x” is the instance
of the adapter. The command will produce output similar to:
The current microcode level for fcs0 is 00011000040041500033
If the ROM Level is
less than 00011000040041500033 you should update the microcode.
NOTE: The instructions that follow are
specific Linux commands. Linux commands are CASE (lower and upper) SENSITIVE,
and must be entered exactly as shown, including filenames.
1) List the Ethernet interfaces for PCIe2 10Gb
4-Port FCoE Adapter adapters installed in the system by typing:
ifconfig
-a | grep eth
Note: All of the Ethernet
adapters that are installed will be displayed. They will be listed as eth{n},
where {n} is 0, 1, etc.
2) For each adapter listed,
use ethtool -i eth{n} to determine which Ethernet interfaces are associated
with the PCIe2 10Gb 4-Port FCoE Adapter. If driver name is “be2net”, that
interface is for PCIe2 10Gb 4-Port FCoE Adapter. For example:
ethtool
-i eth{n}
Where {n} is the instance
of the adapter. The command will produce output similar to:
ethtool -i eth1
driver: be2net
version: 2.102.426r
firmware version:
11.4.415.33
bus-info: 0001:01:00.0
If driver name is “be2net”,
eth1 is one of PCIe2 10Gb 4-Port FCoE Adapter interfaces. firmware-version is
“11.4.415.33”.
If firmware-version is not
11.4.415.33 or higher, on any of the adapters listed with a be2net driver; you
should update the microcode
NOTE: The instructions that
follow are specific AIX commands. AIX commands are CASE (lower and upper)
SENSITIVE, and must be entered exactly as shown, including
filenames.
1) Make two directories on
your AIX system to receive the RPM format file.
Enter: "mkdir
/tmp/microcode"
and then
create this directory
Enter: "mkdir /tmp/microcode/RPM"
2) Transfer the RPM format
file to the /tmp/microcode/RPM directory (using "Save as ...").
Change to that directory, "cd /tmp/microcode/RPM".
You'll see the filename for
the RPM file.
"rpm -ihv --ignoreos df1060e214101004.00011000040041500033.aix.rpm"
3) For AIX: The
microcode files will be added to /etc/microcode/.
NOTE:
- "/etc/microcode" is a symbolic link to
"/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or
file then the user will be prompted for a new location.
Use this method to download the new
microcode to a Linux system:
NOTE: The instructions that follow are
specific Linux commands. Linux commands are CASE (lower and upper) SENSITIVE,
and must be entered exactly as shown, including filenames.
1)
Transfer the RPM format file to the /tmp directory(using “Save as....”).
You will see the filename for the RPM file.
2)
Install rpm on your Linux system by typing:
rpm -ivh /tmp/df1060e214101004.00011000040041500033.Linux.rpm
3) The microcode package will install the
firmware images in the /lib/firmware directory. If a message is displayed
saying the "package <package_name> is already installed", you
will need to uninstall the listed rpm package. On the command line type:
rpm -e <package_name>
where <package_name>
is the name of the package that was returned in the message. Return to Step 2
and attempt to install the file again.
4) The microcode file will be installed to the
/lib/firmware directory. File name：df1060e214101004.00011000040041500033
To obtain information how
to burn a CD-ROM and run the Discovery Tool for an AIX or Linux System please
go to: http://www14.software.ibm.com/support/customercare/mds/
1) After running the
Discovery Tool successfully the "/tmp/microcode/RPM" directory was
created and your rpm files are copied from the CD-ROM.
2) Change to that
directory, "cd /tmp/microcode/RPM".
3) Unpack the file by
executing the instructions below:
Enter the commands:
rpm -ihv --ignoreos df1060e214101004.00011000040041500033.aix.rpm
4) One microcode file will
be copied to "/etc/microcode".
File Name:
df1060e214101004.00011000040041500033
NOTE:
- "/etc/microcode" is a symbolic link to
"/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or
file then the user will be prompted for a new location.
Please verify the file size
and checksum of the raw microcode files matches what is listed below.
For AIX :
Please verify the file size
and checksum
# ls -l df1060e214101004.00011000040041500033
-rwxr-xr-x 1
root system 10970832 Sept 10 16:47 df1060e214101004.00011000040041500033
# sum df1060e214101004.00011000040041500033
34419 10714 df1060e214101004.00011000040041500033
ls -l /lib/firmware/df1060e214101004.00011000040041500033 to verify file size is
10970832
sum /lib/firmware/df1060e214101004.00011000040041500033 to verify Checksum is 34419
Microcode download must be
performed separately for each Adapter under each Logical Partitions (LPAR's).
1) Stop all applications that use
this interface/adapter
2) Detach the Adapter’s
interface (not always needed, but it is a good practice to do this step )
a) Before detaching the interface,
record the ip address and any other pertinent information that was configured
on the Adapter. This information may be
needed if the microcode update overwrites this section on the Adapter.
b) Important Note: If SEA
or Etherchannel is used, they need to be detached and unconfigured before
perform FW update.
For example: ent4 is SEA
#
ifconfig en4 detach
# rmdev
-Rl ent4
3) You only
need to pick one fcs# per adapter for microcode update. Diag will update
microcode for all ports.
Note: in step 3) you might need to perform “rmdev -Rl
fscsi# “ to remove child devices to the
fcs#. Fcs# needs to be Available for fw download.
1) At the command line type
“diag”
2) Select the "Task Selection" from diagnostics menu.
3) Select "Microcode Tasks” then select “Download Microcode" from the
menu.
4) Select all of the PCIe2 10Gb 4-Port FCoE Adapters that need to be updated from the list of devices by using the
arrow keys to highlight the entry and pressing "Enter" to mark
it. The devices will be displayed as fcs#, where # is the number of
the device. Press "F7" when you are done marking all the adapters
you want to flash.
5) If a source selection menu is displayed, Select “/etc/microcode”.
6) A dialogue box may be displayed on screen. It will state that the
current microcode level on the adapter is not in the /etc/microcode
directory. This is acceptable because the adapter will reject any
incorrect code. Press "Enter" to continue.
7) Select 00011000040041500033 level and press "Enter" to flash the adapter.
8) The following message will appear on the screen when download is completed:
"Microcode download complete successfully. The current microcode
level ... Please run diagnostics on the adapter to ensure
that it is functioning properly."
9) If you selected more than one adapter to update, then steps 6-9 will repeat
until all adapters are updated.
10) Exit diagnostics.
1) Verify the code level
is 00011000040041500033 by typing "lsmcode -cd
fcs#" or “lsmcode -cd ent#” for each Fiber Channel or Ethernet adapter
updated. <Where # is the instance of the adapter>
2)
Re-configure/Verify the adapter’s information through
the “smit” menus if necessary.
Example #lscfg
outputs for devices on this adapter
#
lscfg -vl ent0
ent0 U78D5.ND1.CSS241E-P1-C5-C1-T1 PCIe3 10GbE
SFP+ SR 4-port Converged Network Adapter (df1020e214100f04)
PCIe2 4-port (2 10GbE+2 1GbE) FCoE+NIC
SRIOV Adapter, NIC PF:
Part Number.................00E3497
Serial Number...............1A84900086
EC Level....................D77535
Customer Card ID Number.....2B93
Feature Code/Marketing ID...EN0H
Network
Address.............00109B5CE6BB
ROM Level.(alterable).......00011000040041500033
Hardware Location
Code......U78D5.ND1.CSS241E-P1-C5-C1-T1
# lscfg -vl ent1
ent1 U78D5.ND1.CSS241E-P1-C5-C1-T2 PCIe3 10GbE
SFP+ SR 4-port Converged Network Adapter (df1020e214100f04)
PCIe2 4-port (2 10GbE+2 1GbE) FCoE+NIC
SRIOV Adapter, NIC PF:
Part Number.................00E3497
Serial Number...............1A84900086
EC Level....................D77535
Customer Card ID Number.....2B93
Feature Code/Marketing ID...EN0H
Network
Address.............00109B5CE6BC
ROM
Level.(alterable).......00011000040041500033
Hardware Location
Code......U78D5.ND1.CSS241E-P1-C5-C1-T2
# lscfg -vl ent2
ent2 U78D5.ND1.CSS241E-P1-C5-C1-T3 PCIe3
100/1000 Base-TX 4-port Converged Network Adapter (df1020e214103c04)
PCIe2 4-port (2 10GbE+2 1GbE) FCoE+NIC
SRIOV Adapter, NIC PF:
Part Number.................00E3497
Serial Number...............1A84900086
EC Level....................D77535
Customer Card ID Number.....2B93
Feature Code/Marketing ID...EN0H
Network
Address.............00109B5CE6BD
ROM
Level.(alterable).......00011000040041500033
Hardware Location
Code......U78D5.ND1.CSS241E-P1-C5-C1-T3
# lscfg -vl ent3
ent3 U78D5.ND1.CSS241E-P1-C5-C1-T4 PCIe3
100/1000 Base-TX 4-port Converged Network Adapter (df1020e214103c04)
PCIe2 4-port (2 10GbE+2 1GbE) FCoE+NIC
SRIOV Adapter, NIC PF:
Part Number.................00E3497
Serial Number...............1A84900086
EC Level....................D77535
Customer Card ID Number.....2B93
Feature Code/Marketing ID...EN0H
Network
Address.............00109B5CE6BE
ROM Level.(alterable).......00011000040041500033
Hardware Location
Code......U78D5.ND1.CSS241E-P1-C5-C1-T4
# lscfg -vl fcs0
fcs0 U78D5.ND1.CSS241E-P1-C5-C1-T1 PCIe3 10Gb
4-Port FCoE Adapter (df1060e214101004)
Part Number.................00E3497
Serial Number...............1A84900086
EC Level....................D77535
Customer Card ID Number.....2B93
Manufacturer................001A
FRU Number..................00E3498
Network
Address.............100000109B5CE6BF
Device Specific.(Z0)........0000000D
Device Specific.(Z1)........00000030
Device Specific.(Z2)........00000000
Device Specific.(Z3)........08090000
Device Specific.(Z4)........00003001
Device Specific.(Z5)........2E343135
Device Specific.(Z6)........2E343135
Device Specific.(Z7)........C0022B40
Device Specific.(Z8)........200000109B5CE6BF
Device Specific.(Z9)........11.4.415.33
Device Specific.(ZA)........11.4.415.33
Device Specific.(ZB)........30313233
Device Specific.(ZC)........00040000
Device Specific.(ZD)........000000FF
Hardware Location
Code......U78D5.ND1.CSS241E-P1-C5-C1-T1
# lscfg -vl fcs1
fcs1 U78D5.ND1.CSS241E-P1-C5-C1-T2 PCIe3 10Gb
4-Port FCoE Adapter (df1060e214101004)
Part Number.................00E3497
Serial Number...............1A84900086
EC Level....................D77535
Customer Card ID Number.....2B93
Manufacturer................001A
FRU Number..................00E3498
Network
Address.............100000109B5CE6C0
Device Specific.(Z0)........0000000D
Device Specific.(Z1)........00000030
Device Specific.(Z2)........00000000
Device Specific.(Z3)........08090000
Device Specific.(Z4)........00013101
Device Specific.(Z5)........2E343135
Device Specific.(Z6)........2E343135
Device Specific.(Z7)........C0022B40
Device
Specific.(Z8)........200000109B5CE6C0
Device Specific.(Z9)........11.4.415.33
Device
Specific.(ZA)........11.4.415.33
Device Specific.(ZB)........30313233
Device Specific.(ZC)........00040000
Device Specific.(ZD)........000000FF
Hardware Location
Code......U78D5.ND1.CSS241E-P1-C5-C1-T2
1)
Use this
series of Linux commands to determine which of any installed Network adapters
are PCIe2 10Gb 4-Port FCoE Adapter:
for i in `ls /sys/class/net | grep
eth` ; do echo -n "${i}: "; ethtool -i ${i} | grep bus-info; done
Output
will display the association between eth{n} devices and PCI bus device numbers,
nnnn:nn:nn.n
Output
will list PCI bus device numbers for the PCIe2
10Gb 4-Port FCoE Adapter functions, nn:nn.n
This
example shows the four ports of one PCIe2 10Gb 4-Port FCoE
Adapter. Firmware need only be updated on
the first port of each adapter,
2)
Stop all
applications that use the interface/adapter to be updated
3)
Configure
the adapter’s interface, eth1 in this example.
1)
Using
the eth{n} configured in previous step use ethtool --flash to
update the firmware.
ethtool --flash eth{n}
df1060e214101004.00011000040041500033
For example,
dmesg command will show the following messages:
be2net
0000:01:00.0: Flashing firmware file df1060e214101004.00011000040041500033
be2net 0000:01:00.0: System reboot required for new FW to be active
be2net 0000:01:00.0: Firmware flashed successfully
These
messages show flashing is completed successfully.
3)
Reboot
the OS instance to activate the updated firmware.
4)
Verify
firmware has been activated
Using
same eth{n} interface used for ethtool --flash
command, type: