Power9 System Firmware

Applies to:   9080-M9S

This document provides information about the installation of Licensed Machine or Licensed Internal Code, which is sometimes referred to generically as microcode or firmware.

Contents

1.0 Systems Affected

This package provides firmware for Power Systems E980 (9080-M9S) servers only.

The firmware level in this package is:

1.1 Minimum HMC Code Level

This section is intended to describe the "Minimum HMC Code Level" required by the System Firmware to complete the firmware installation process. When installing the System Firmware, the HMC level must be equal to or higher than the "Minimum HMC Code Level" before starting the system firmware update.  If the HMC managing the server targeted for the System Firmware update is running a code level lower than the "Minimum HMC Code Level" the firmware update will not proceed.

The Minimum HMC Code levels for this firmware for HMC x86,  ppc64 or ppc64le are listed below.

x86 -  This term is used to reference the legacy HMC that runs on x86/Intel/AMD hardware for the Virtual HMC that can run on the Intel hypervisors (KVM, XEN, VMWare ESXi).
ppc64 or ppc64le - describes the Linux code that is compiled to run on Power-based servers or LPARS (Logical Partitions)
The Minimum HMC level supports the following HMC models:
x86 - KVM, XEN, VMWare ESXi (6.0/6.5)
ppc64le - 7063-CR1,vHMC on PowerVM (POWER8 and POWER9 systems

For information concerning HMC releases and the latest PTFs,  go to the following URL to access Fix Central:
http://www-933.ibm.com/support/fixcentral/

For specific fix level information on key components of IBM Power Systems running the AIX, IBM i and Linux operating systems, we suggest using the Fix Level Recommendation Tool (FLRT):
http://www14.software.ibm.com/webapp/set2/flrt/home


NOTES:
                -You must be logged in as hscroot in order for the firmware installation to complete correctly.
                - Systems Director Management Console (SDMC) does not support this System Firmware level.

2.0 Important Information

NovaLink levels earlier than "NovaLink 1.0.0.16 Feb 2020 release" with partitions running certain SR-IOV capable adapters is NOT supported at this firmware release

NovaLink levels earlier than "NovaLink 1.0.0.16 Feb 2020 release" do not support IO adapter FCs EC2R/EC2S, EC2T/EC2U, EC3L/EC3M, EC66/EC67 with FW930 and later.   If the adapter was already in use with FW910/920 at an older NovaLink level, upgrading to FW930/940 will result in errors in NovaLink and PowerVC which causes the loss of any management operation via NovaLink / PowerVC combination.  Upgrading systems in this configuration is not supported at the older NovaLink levels.  If the system is required to be at FW930/940 or was shipped with FW930/940, NovaLink must first be updated to "NovaLink 1.0.0.16 Feb 2020 release"  or later.

Boot adapter microcode requirement

Update all adapters which are boot adapters, or which may be used as boot adapters in the future, to the latest microcode from IBM Fix Central.  The latest microcode will ensure the adapters support the new Firmware Secure Boot feature of Power Systems. This requirement applies when updating system firmware from a level prior to FW940 to levels FW940 and later.
The latest adapter microcode levels include signed boot driver code. If a boot-capable PCI adapter is not installed with the latest level of adapter microcode, the partition which owns the adapter will boot, but error logs with SRCs BA5400A5 or BA5400A6 will be posted.  Once the adapter(s) are updated, the error logs will no longer be posted.

Downgrading firmware from any given release level to an earlier release level is not recommended

Firmware downgrade warnings:
1) Adapter feature codes (#EC2R/#EC2S/#EC2T/#EC2U and #EC3L/#EC3M and #EC66/EC67) when configured in SR-IOV shared mode in FW930 or later, even if originally configured in shared mode in a pre-FW930 release, may not function properly if the system is downgraded to a pre-FW930 release. The adapter should be configured in dedicated mode first (i.e. take the adapter out of SR-IOV shared mode) before downgrading to a pre-FW930 release.
2) If partitions have been run in POWER9 compatibility mode in FW940, a downgrade to an earlier release (pre-FW940) may cause a problem with the partitions starting.  To prevent this problem, the "server firmware" settings must be reset by rebooting partitions in "Power9_base" before doing the downgrade.

If you feel that it is necessary to downgrade the firmware on your system to an earlier release level, please contact your next level of support.

NovaLink issue with remote start and inactive partition migration operations

There is a known issue with remote restart and inactive partition migration operations initiated from NovaLink when the source system is running this firmware level. Remote restart and inactive partition migration functionality initiated from an HMC is not impacted by this issue.   When attempting these operations with PowerVC on a NovaLink managed environment, a PVME01040100-0004 with Internal error PVME01038003 may be encountered on the target system.  This issue is planned to be addressed in a future FW950.00 Service Pack.  Systems running with a previous firmware level should remain at that level until a FW950 Service Pack is released which addresses the issue with remote restart and inactive partition migration operations initiated from Novalink. If you have systems shipped with FW950.00 and require more immediate support for remote restart or inactive partition migration operations initiated from NovaLink, please contact IBM support.

2.1 IPv6 Support and Limitations

IPv6 (Internet Protocol version 6) is supported in the System Management Services (SMS) in this level of system firmware. There are several limitations that should be considered.

When configuring a network interface card (NIC) for remote IPL, only the most recently configured protocol (IPv4 or IPv6) is retained. For example, if the network interface card was previously configured with IPv4 information and is now being configured with IPv6 information, the IPv4 configuration information is discarded.

A single network interface card may only be chosen once for the boot device list. In other words, the interface cannot be configured for the IPv6 protocol and for the IPv4 protocol at the same time.

2.2 Concurrent Firmware Updates

Concurrent system firmware update is supported on HMC Managed Systems only.

Ensure that there are no RMC connections issues for any system partitions prior to applying the firmware update.  If there is a RMC connection failure to a partition during the firmware update, the RMC connection will need to be restored and additional recovery actions for that partition will be required to complete partition firmware updates.

2.3 Memory Considerations for Firmware Upgrades

Firmware Release Level upgrades and Service Pack updates may consume additional system memory.
Server firmware requires memory to support the logical partitions on the server. The amount of memory required by the server firmware varies according to several factors.
Factors influencing server firmware memory requirements include the following:
Generally, you can estimate the amount of memory required by server firmware to be approximately 8% of the system installed memory. The actual amount required will generally be less than 8%. However, there are some server models that require an absolute minimum amount of memory for server firmware, regardless of the previously mentioned considerations.

Additional information can be found at:
https://www.ibm.com/support/knowledgecenter/9080-M9S/p9hat/p9hat_lparmemory.htm

2.4 SBE Updates

Power 9 servers contain SBEs (Self Boot Engines) and are used to boot the system.  SBE is internal to each of the Power 9 chips and used to "self boot" the chip.  The SBE image is persistent and is only reloaded if there is a system firmware update that contains a SBE change.  If there is a SBE change and system firmware update is concurrent, then the SBE update is delayed to the next IPL of the CEC which will cause an additional 3-5 minutes per processor chip in the system to be added on to the IPL.  If there is a SBE change and the system firmware update is disruptive, then SBE update will cause an additional 3-5 minutes per processor chip in the system to be added on to the IPL.  During the SBE update process, the HMC or op-panel will display service processor code C1C3C213 for each of the SBEs being updated.  This is a normal progress code and system boot should be not be terminated by the user. Additional time estimate can be between 12-20 minutes per drawer or up to 48-80 minutes for maximum configuration.

The SBE image is updated with this service pack.

3.0 Firmware Information

Use the following examples as a reference to determine whether your installation will be concurrent or disruptive.

For systems that are not managed by an HMC, the installation of system firmware is always disruptive.

Note: The concurrent levels of system firmware may, on occasion, contain fixes that are known as Deferred and/or Partition-Deferred. Deferred fixes can be installed concurrently, but will not be activated until the next IPL. Partition-Deferred fixes can be installed concurrently, but will not be activated until a partition reactivate is performed. Deferred and/or Partition-Deferred fixes, if any, will be identified in the "Firmware Update Descriptions" table of this document. For these types of fixes (Deferred and/or Partition-Deferred) within a service pack, only the fixes in the service pack which cannot be concurrently activated are deferred.

Note: The file names and service pack levels used in the following examples are for clarification only, and are not necessarily levels that have been, or will be released.

System firmware file naming convention:

01VHxxx_yyy_zzz

NOTE: Values of service pack and last disruptive service pack level (yyy and zzz) are only unique within a release level (xxx). For example, 01VH900_040_040 and 01VH910_040_045 are different service packs.

An installation is disruptive if:

            Example: Currently installed release is 01VH900_040_040, new release is 01VH910_050_050.

            Example: VH910_040_040 is disruptive, no matter what level of VH910 is currently installed on the system.

            Example: Currently installed service pack is VH910_040_040 and new service pack is VH910_050_045.

An installation is concurrent if:

The release level (xxx) is the same, and
The service pack level (yyy) currently installed on the system is the same or higher than the last disruptive service pack level (zzz) of the service pack to be installed.

Example: Currently installed service pack is VH910_040_040, new service pack is VH910_041_040.

3.1 Firmware Information and Description

 
Filename Size Checksum md5sum
01VH950_045_045.rpm 173594702
01179
 7305aae7a56a54c49b1035cd6d57f9d4

Note: The Checksum can be found by running the AIX sum command against the rpm file (only the first 5 digits are listed).
ie: sum 01VH950_045_045.rpm

VH950
For Impact, Severity and other Firmware definitions, Please refer to the below 'Glossary of firmware terms' url:
http://www14.software.ibm.com/webapp/set2/sas/f/power5cm/home.html#termdefs

The complete Firmware Fix History for this Release Level can be reviewed at the following url:
http://download.boulder.ibm.com/ibmdl/pub/software/server/firmware/VH-Firmware-Hist.html
VH950_045_045 / FW950.00

11/23/20		 Impact:  New      Severity:  New

GA Level with key features included listed below
  • All features and fixes from the FW930.30 and FW940.20 service packs (and below) are included in this release.
New Features and Functions
  • Host firmware support for anti-rollback protection.  This feature implements firmware anti-rollback protection as described in NIST SP 800-147B "BIOS Protection Guidelines for Servers".  Firmware is signed with a "secure version".  Support added for a new menu in ASMI called "Host firmware security policy" to update this secure version level at the processor hardware.  Using this menu, the system administrator can enable the "Host firmware secure version lock-in" policy, which will cause the host firmware to update the "minimum secure version" to match the currently running firmware. Use the "Firmware Update Policy" menu in ASMI to show the current "minimum secure version" in the processor hardware along with the "Minimum code level supported" information. The secure boot verification process will block installing any firmware secure version that is less than the "minimum secure version" maintained in the processor hardware.
    Prior to enabling the "lock-in" policy, it is recommended to accept the current firmware level.
    WARNING: Once lock-in is enabled and the system is booted, the "minimum secure version" is updated and there is no way to roll it back to allow installing firmware releases with a lesser secure version.
    Note:  If upgrading from FW930.30 or FW940.20, this feature is already applied.
  • This server firmware level includes the SR-IOV adapter firmware level 11.4.415.33 for the following Feature Codes and CCINs: #EN15/EN16 with CCIN 2CE3, #EN17/EN18 with CCIN 2CE4, #EN0H/EN0J with CCIN 2B93, #EN0M/EN0N with CCIN 2CC0, and #EN0K/EN0L with CCIN 2CC1.
  • This server firmware includes the SR-IOV adapter firmware level 1x.25.6100 for the following Feature Codes and CCINs: #EC2R/EC2S with CCIN 58FA; #EC2T/EC2U with CCIN 58FB; #EC3L/EC3M with CCIN 2CEC; and #EC66/EC67 with CCIN 2CF3.
  • Support added for IBM i 7.1 (Tech Refresh 11 + PTFs) for restricted I/O only.
  • Support for PCIe4 x8 1.6/3.2/6.4 TB NVMe Adapters that are Peripheral Component Interconnect Express (PCIe) Generation 4 (Gen4) x8 adapters with the following feature codes and CCINs:
    #EC7A/#EC7B with CCIN 594A ; #EC7C/#EC7D with CCIN 594B; and #EC7E/#EC7F with CCIN 594C for AIX/Linux.
    #EC7J/#EC7K with CCIN 594A ; #EC7L/#EC7M with CCIN 594B; and #EC7N/#EC7P with CCIN 594C for IBM i.
  • PowerVM boot support for AIX for NVMe over Fabrics (NVMf) for 32Gb Fibre Channel.  Natively attached adapters are supported with the following feature codes and CCINs: #EN1A/#EN1B with CCIN 578F.
  • Support added for a PCIe2 2-Port USB 3.0 adapter with the following feature codes and CCIN: #EC6J/#EC6K with CCIN 590F.
  • Support added for dedicated processor partitions in IBM Power Enterprise Pools (PEP) 2.0.  Previously, systems added to PEP 2.0 needed to have all partitions as shared processor partitions.
  • Support added for SR-IOV Hybrid Network Virtualization (HNV) for Linux.   This capability allows a Linux partition to take advantage of the efficiency and performance benefits of SR-IOV logical ports and participate in mobility operations such as active and inactive Live Partition Mobility (LPM) and Simplified Remote Restart (SRR).  HNV is enabled by selecting a new Migratable option when an SR-IOV logical port is configured. The Migratable option is used to create a backup virtual device.  The backup virtual device must be a Virtual Ethernet adapter (virtual Network Interface Controller (vNIC) adapter not supported as a backup device). In addition to this firmware, HNV support in a production environment requires HMC 9.1.941.0 or later, RHEL 8., SLES 15, and VIOS 3.1.1.20 or later.
  • Enhanced  Dynamic DMA Window (DDW) for I/O adapter slots to enable the OS to use 64KB TCEs.   The OS supported is Linux RHEL 8.3 LE.
  • PowerVM support for the Platform KeyStore (PKS) for partitions.  PowerVM has added new h-call interfaces allowing the partition to interact with the Platform KeyStore that is maintained by PowerVM.  This keystore can be used by the partition to store items requiring confidentiality or integrity like encryption keys or certificates.
    Note:  The total amount of PKS for the system is limited to 1 MB across all the partitions for FW950.00.
System firmware changes that affect all systems
  • HIPER/Pervasive: A problem was fixed for a system checkstop with an SRC BC14E540 logged that can occur during certain SMP cable failure scenarios.
  • HIPER/Pervasive: A problem was fixed for soft error recovery not working in the  DPSS (Digital Power Subsystem Sweep) programmable power controller that results in the DPSS being called out as a failed FRU.  However, the DPSS is recovered on the next IPL of the system.
  • HIPER/Pervasive:  A problem was fixed to be able to detect a failed PFET sensing circuit in a core at runtime, and prevent a system fail with an incomplete state when a core fails to wake up.  The failed core is detected on the subsequent IPL.  With the fix. a core is called out with the PFET failure with SRC BC13090F and hardware description "CME detected malfunctioning of PFET headers." to isolate the error better with a correct callout.
  • A problem was fixed for system UPIC cable validation not being able to detect cross-plugged UPIC cables.  If the cables are plugged incorrectly and there is a need for service, modifying the wrong FRU locations can have adverse effects on the system, including system outage. The cable status that is displayed is the result of the last cable validation that occurred. Cable validation occurs automatically during system power on.
    Note:  If upgrading from FW930.30, this fix is already applied.
  • A problem was fixed for a VIOS, AIX, or Linux partition hang during an activation at SRC CA000040.  This will occur on a system that has been running more than 814 days when the boot of the partition is attempted if the partitions are in POWER9_base or POWER9 processor compatibility mode.
    A workaround to this problem is to re-IPL the system or to change the failing partition to POWER8 compatibility mode.
    Note:  If upgrading from FW930.30, this fix is already applied.
  • A problem was fixed for a security vulnerability for the Self Boot Engine (SBE). The SBE can be compromised from the service processor to allow injection of malicious code. An attacker that gains root access to the service processor could compromise the integrity of the host firmware and bypass the host firmware signature verification process. This compromised state can not be detected through TPM attestation.  This is Common Vulnerabilities and Exposures issue number CVE-2021-20487.

4.0 How to Determine The Currently Installed Firmware Level

You can view the server's current firmware level on the Advanced System Management Interface (ASMI) Welcome pane. It appears in the top right corner. Example: VH920_123.

5.0 Downloading the Firmware Package

Follow the instructions on Fix Central. You must read and agree to the license agreement to obtain the firmware packages.

Note: If your HMC is not internet-connected you will need to download the new firmware level to a USB flash memory device or ftp server.

6.0 Installing the Firmware

The method used to install new firmware will depend on the release level of firmware which is currently installed on your server. The release level can be determined by the prefix of the new firmware's filename.

Example: VHxxx_yyy_zzz

Where xxx = release level

Instructions for installing firmware updates and upgrades can be found at https://www.ibm.com/support/knowledgecenter/9080-M9S/p9eh6/p9eh6_updates_sys.htm

IBM i Systems:

For information concerning IBM i Systems, go to the following URL to access Fix Central: 
http://www-933.ibm.com/support/fixcentral/

Choose "Select product", under Product Group specify "System i", under Product specify "IBM i", then Continue and specify the desired firmware PTF accordingly.

7.0 Firmware History

The complete Firmware Fix History (including HIPER descriptions)  for this Release level can be reviewed at the following url:
http://download.boulder.ibm.com/ibmdl/pub/software/server/firmware/VH-Firmware-Hist.html

8.0 Change History

Date
 Description
September 15, 2021
 Added a defect statement:
A problem was fixed for a security vulnerability for the Self Boot Engine (SBE).  The SBE can be compromised from the service processor to allow injection of malicious code. An attacker that gains root access to the service processor could compromise the integrity of the host firmware and bypass the host firmware signature verification process. This compromised state can not be detected through TPM attestation.  This is Common Vulnerabilities and Exposures issue number CVE-2021-20487.
March 30, 2021
 Updated HMC model 7042-CR9 support in section 1.1 Minimum HMC Code Level.
March 12, 2021
 Fix descripton text updated and now classified as HIPER/Pervasive.
January 5, 2021 Updated NovaLink warning in section 2.0 Important Information.
December 16, 2020 Firmware Level VH950_045_045 / FW950.00:
Fix descripton additions for level VH950_045_045 / FW950.00.
Fix descripton text updated and now classified as HIPER/Pervasive.
Updated HMC code level and models in section 1.1 Minimum HMC Code Level.