PCIe4 NVMe Flash Adapter (EC7A, EC7C, EC7E) Microcode README IBM POWER Systems Non-Volatile Memory PCIe4 x8 1.6TB SSD NVMe Adapter (CCIN 594A) Non-Volatile Memory PCIe4 x8 3.2TB SSD NVMe Adapter (CCIN 594B) Non-Volatile Memory PCIe4 x8 6.4TB SSD NVMe Adapter (CCIN 594C) AIX & Linux Feature Codes: EC7A, EC7B, EC7C, EC7D, EC7E, EC7F Microcode Update    DATE:  May 22, 2022   Table of Contents 1.0 Microcode Change History 2.0 General Information 2.1 Installation Time 2.2 Machines Affected 2.3 Operating System Requirements 3.0 Determine the Current Microcode Level for AIX & Linux 4.0 Downloading and Unpacking the Firmware Update Package 4.1 Downloading the RPM Format File to the Target Server 5.0 Verifying microcode before download and notes 5.1 Microcode Download Procedure-NVMe Adapters for AIX 5.2 Microcode Download Procedure-NVMe Adapters for Linux 6.0 Microcode Readme Change History ===================================================================================== 1.0 Microcode Change History:   53503437 (SP47) Impact:    New Severity:  New GA Level   2.0 General information This subject microcode is to be used on: PCIe4 1.6TB NVMe Flash Adapter (CCIN 594A), FC (Low Profile/High Profile): EC7A / EC7B PCIe4 3.2TB NVMe Flash Adapter (CCIN 594B), FC (Low Profile/High Profile): EC7C / EC7D PCIe4 6.4TB NVMe Flash Adapter (CCIN 594C), FC (Low Profile/High Profile): EC7E / EC7F 2.1 Installation Time: Approximately 30 minutes   2.2 Machine's Affected: 9009-22A, 9009-22L, 9009-41A, 9009-42A, 9223-22H, 9223-42H, 9009-22G, 9009-41G, 9009-42G, 9040-MR9, 9080-M9S See also: Universal Sales Manual or other relevant documents for current listings of supported systems for these adapters   2.3 Operating System Requirements: If you are installing a new feature, ensure that you have the software that is required to support the new feature and that you determine whether any prerequisites must be met for this feature.  Consult the IBM sales manual for your system model to determine the AIX, VIOS, or Linux release level requisites to support the feature in the system.  To check for the feature prerequisites, see the IBM Prerequisite website (https://www14.software.ibm.com/support/customercare/iprt/home).    3.0 Determine the Current Microcode Level for AIX & Linux: These steps assume you are logged in with root permissions and a PCIe4 NVMe Flash Adapter, Feature Code EC7A, EC7B, EC7C, EC7D, EC7E, or EC7F is installed.   Prerequisites for AIX: There is no additional prerequisite software needed beyond the supported level of AIX.   Prerequisites for Linux: Use the ibm-power-repo to configure the repositories and then use yum to download thenvme-cli package. http://www14.software.ibm.com/support/customercare/sas/f/lopdiags/home.html   1. Download the IBM Linux on Power Tools Repository configuration RPM file available at: http://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/yum.html   2. Copy the file to the target system and install the RPM # rpm -ivh ibm-power-repo-latest.noarch.rpm   3. To add the Tools Repository to the package management configuration, run the following command as root user (or with root privilege) and accept the Licensing agreement. # /opt/ibm/lop/configure   Instructions for installing nvme-cli tool using Red Hat Enterprise Linux   To list the currently enabled repositories, enter: # yum repolist enabled   Install the nvme-cli package, enter: # yum install nvme-cli   Checking NVME device firmware for AIX: To list NVMe devices on your system and view firmware level,   1) Type:  ls /dev | grep nvme      nvme0      nvme1   2)  List details of the nvme device(s) by:        Type:  lscfg -vl nvme#                 # lscfg -vl nvme0   nvme1            U78D4.ND1.2852224-P1-C2  PCIe4 x8 NVMe Flash Adapter         PCIe4 1.6TB NVMe Flash Adapter x8       :         Part Number.................02DE959         EC Level....................P46698         FRU Number..................02DE956         Product Specific.(AN).......02DE958         Feature Code/Marketing ID...EC7B         Customer Card ID Number.....594A         Serial Number...............AFYDJ025         Product Specific.(Z0).......10140682         Product Specific.(Z1).......3.0         Product Specific.(Z2).......1600         Product Specific.(Z3).......8.76         Product Specific.(Z4).......1         Product Specific.(Z5).......00         Product Specific.(Z6).......A180010D         Product Specific.(Z7).......0YA         Product Specific.(Z8).......3023         Product Specific.(Z9).......08         Product Specific.(ZA).......0008         Product Specific.(ZB).......0040         Product Specific.(ZC).......00         Manufacture ID..............S4WANE0N300071         ROM Level.(alterable).......REV.SP47         Product Specific.(RV).......s         Hardware Location Code......U78D4.ND1.2852224-P1-C2   The firmware level is listed in the field:  ROM Level (alterable).   If the Firmware Version is less than REV.SP47 then you should update the microcode.     Checking NVMe device firmware on Linux: After nvme-cli is installed on the system, check the current microcode level for the adapter enter the following command: # nvme list   Note: "nvme list" command uses libudev-devel, if it doesn't work, may need to install ?libudev-devel"   If the FW Rev is different from ?REV.SP47? install this update. 1) Type "nvme list" 2) The column ?FW Rev? will list the firmware version.  For example:   Node             SN                  Model                                    Namespace        Usage                    Format             FW Rev  ---------------- ---------------     --------------------------------------   ------------     ----------------------   ----------------   ---------------- /dev/nvme1n1     S4WANE0N300071      PCIe4 1.6TB NVMe Flash Adapter x8         1                  1.60  TB /   1.60  TB     4 KiB +  0 B     Rev.SP47   If the Firmware Version is less than REV.SP47 then you should update the microcode. 4.0 Downloading and Unpacking the Firmware Update Package NOTE:  For RHEL, the image will be the RPM format file.   1. Download feature code EC7A, EC7B, EC7C, EC7D, EC7E, or EC7F package from FixCentral, and transfer the file, to a temporary directory on the target server such as /tmp.   2. Install the package   (AIX) rpm -ihv --ignoreos --force pci.144dA824.SP47-53503437-1.aix6.1.noarch.rpm   (Linux) rpm ?ivh pci.144dA824.SP47-53503437-1.Linux.noarch.rpm   3. The microcode package extracts the firmware file to the following directory: ·       Linux:  /lib/firmware ·       AIX:    /etc/microcode   4. Verify the microcode image in Section 5.0 5.0 Verifying microcode before download and notes NOTE: Microcode download must be performed separately for each Adapter bus under distinct Logical Partitions (LPAR's).               Please save current and older versions of the microcode update files in case you need to restore your system. NOTE: Microcode Download CAN be performed concurrently. Installation Time:  Approximately 30 min System Time. Please verify the file size and checksum of the raw microcode files matches the below. Filesize:  6093824 Checksum:  29420   For AIX:    " ls -l /etc/microcode/A180010D.53503437" to verify Filesize.    "sum /etc/microcode/A180010D.53503437" to verify Checksum For Linux:    "ls -l /lib/firmware/pci.144dA824.SP47.53503437" to verify Filesize.    "sum /lib/firmware/pci.144dA824.SP47.53503437" to verify Checksum. For systems that have LPAR installed, the microcode file must be placed on all partitions that have aNVMe device in order to update the code.  Follow this procedure for each partition: For AIX:     A)  Log into the target partition.     B)  Type: "cd /etc/microcode"     C)  SFTP to the partition that has the media bay assigned. (ie.  "sftp partition1.ibm.com")     D)  Type: "cd /etc/microcode"     E)  Type: "get A180010D.53503437"     F)  Type: "quit" For Linux:     A)  Log into the target partition.     B)  Type: "cd /lib/firmware"     C)  SFTP to the partition that has the media bay assigned. (ie.  "sftp partition1.ibm.com")     D)  Type: "cd /lib/firmware"     E)  Type: "get pci.144dA824.SP47.53503437"     F)  Type: "quit" 5.1 Microcode Download Procedure-NVMe Adapters for AIX   Procedure: 1) Stop all applications. 2) Type "ls /dev | grep nvme" and press "Enter".  This will list all of the NVMe adapters in your partition 3) To display details for the nvme device (including current firmware level),      type ?lscfg -vl nvme#?.        For example, for device nvme0, type ?lscfg -vl nvme0?   4) For each NVMe adapter to be updated, issue the following commands:      a)  Type ?diag?      b)  Select ?Task Selection?      c)  Select ?Microcode Tasks?      d)  Select ?Download Microcode?      e)  Select the NVMe device to have microcode updated on.             For example, select:  nvme0  and press F7      f)  Select ?/etc/microcode? as location of updated firmware image.  Press ENTER      g) For level to install, select ?53503437?.            The system will take several minutes to update the firmware.            Once completed, the screen will display the following message:     INSTALL MICROCODE                                                                                              nvme0    PCIe4 x8 NVMe Flash Adapter   Installation of the microcode has completed successfully on the following instances of the adapter:   nvme0   The current microcode level is 53503437.   Please run diagnostics on these resources to ensure that the adapter is functioning properly.     Use Enter to continue.     5.2 Microcode Download Procedure-NVMe Adapters for Linux Procedure: 1) Stop all applications. 2) Type "nvme list" and press "Enter".  This will list all of the NVMe adapters in your partition.  In the Node column, the NVMe adapters will be listed as/dev/nvme#n1 where # is the number of the device. 3) For each NVMe adapter to be update, issue the following commands: a)     Type ?nvme fw-log /dev/nvme<#>? and press ?Enter?, where <#> is the number of the device.    For example, for device /dev/nvme0n1, type ?nvme fw-log /dev/nvme0? and press ?Enter?.  This will list the firmware level currently loaded on the adapter as well as the number of slots in the adapter.  These 3 slots are listed as frs1, frs2, and frs3.   b)    Download & activate firmware to the adapter by typing:   ?nvme fw-download /dev/nvme# --fw =/lib/firmware/pci.1014064a.REV.SP47.4d413234? and press ?Enter?, where # is the number of the device.                                i.         nvme fw-download /dev/nvme# --fw =/lib/firmware/pci.144dA824.SP47.53503437 and press ?Enter?, where # is the number of the device.   For example, for device /dev/nvme0, type ?nvme fw-download /dev/nvme0 ?fw =/lib/firmware/pci.144dA824.SP47.53503437? and press ?Enter?.                          ii.         ?nvme fw-activate /dev/nvme# -a 0 ?s 1? where # is the number of the device                      iii.         nvme fw-download /dev/nvme# --fw =/lib/firmware/pci.144dA824.SP47.53503437   (Repeat of step i)                       iv.         ?nvme fw-activate /dev/nvme# -a 0 ?s 2? where # is the number of the device                        v.         nvme fw-download /dev/nvme# --fw =/lib/firmware/pci.144dA824.SP47.53503437   (Repeat of step i)                       vi.         ?nvme fw-activate /dev/nvme# -a 0 ?s 3? where # is the number of the device                     vii.         ?nvme fw-activate /dev/nvme# -a 3 ?s 3? where # is the number of the device                   viii.          ?nvme reset /dev/nvme#? where # is the number of the device   For example, to download & activate the firmware on NVMe device /dev/nvme0, type:                            i.         nvme fw-download /dev/nvme0 --fw =/lib/firmware/pci.144dA824.SP47.53503437                        ii.         nvme fw-activate /dev/nvme0 -a 0 ?s 1                      iii.         nvme fw-download /dev/nvme0 --fw =/lib/firmware/pci.144dA824.SP47.53503437                       iv.         nvme fw-activate /dev/nvme0 -a 0 ?s 2                        v.         nvme fw-download /dev/nvme0 --fw =/lib/firmware/pci.144dA824.SP47.53503437                       vi.         nvme fw-activate /dev/nvme0 -a 0 ?s 3                     vii.         nvme fw-activate /dev/nvme0 -a 3 ?s 3                   viii.          nvme reset /dev/nvme0               6.0 Microcode Readme Change History 2020-11-17 GA microcode level 53503435 (SP47)