IBM Power Systems Scale-out LC Server Firmware
Applies to: LC921(9006-12P) and LC922(9006-22P)
This document provides information about the installation of Licensed Machine or Licensed Internal Code, which is sometimes referred to generically as microcode or firmware.
This package provides firmware for the Power Systems Scale-out LC921 (9006-12P) and LC922 (9006-22P).
The firmware level in this package is:
•V1.16-20180531
These are the following images:
BMC Firmware: SMT_P9_127.bin
PNOR Firmware: P9DSU20180531_IBM_prod_sign.pnor (Production keys)
pUpdate version 2.18 Utility for powerpc: pUpdate
Details on the package binaries are included in section 3.1
This section specifies the "Minimum ipmitool Code Level" required by the System Firmware to perform firmware installations and managing the system. OpenPOWER requires ipmitool level v1.8.15 to execute correctly on the LC server firmware.
Verify your ipmitool level on your Linux workstation using the following commands:
bash-4.1$ ipmitool -V
ipmitool version 1.8.15
If you need to update or add impitool to your Linux workstation , you can compile ipmitool (current level 1.8.15) for Linux as follows from Sourceforge:
1.1.1 Download ipmitool tar from http://sourceforge.net/projects/ipmitool/ to your Linux system
1.1.2 Extract tarball on Linux system
1.1.3 cd to top-level directory
1.1.4 ./configure
1.1.5 make
1.1.6 ipmitool will be under src/ipmitool
The BMC Web GUI is a web-based application that works within a browser. Supported browser levels are shown below with Chrome being the preferred browser:
•Google Chrome Version 46.0.2490.71 m
•Mozilla Firefox version 41.0.3
For specific fix level information on key components of IBM Power Systems servers and Linux operating systems, please refer to the documentation in the IBM Knowledge Center.
Here are the links for the LC921 and LC922 servers:
9006-12P: http://www.ibm.com/support/knowledgecenter/POWER9/p9hdx/9006_12p_landing.htm
9006-22P http://www.ibm.com/support/knowledgecenter/POWER9/p9hdx/9006_22p_landing.htm
On Red Hat Enterprise Linux (RHEL) for PPC, RHEL-Alt 7.5, The Trusted Platform Module (TPM) device driver is not loaded automatically at boot time. Without this driver, the TPM device will not be accessible.
This affects any user-space application needing to access the TPM, as well as kernel security functions, such as the Integrity Measurement Architecture subsystem (IMA) in the Linux kernel. Without the TPM driver loaded, IMA will be unable to record trusted measurements to the TPM.
To load the driver manually, as root:
# modprobe tpm_i2c_nuvoton
To load the driver automatically at boot time:
# echo "tpm_i2c_nuvoton" > /etc/modules-load.d/tpm.conf"
The TPM device driver will be integrated as a built-in kernel module in a future release 7 of RHEL-Alt. Once this is done, it will be loaded automatically and this procedure will no longer be necessary
Downgrading firmware from any given release level to an earlier release level is not recommended.
If you feel that it is necessary to downgrade the firmware on your system to an earlier release level, please contact your next level of support.
Concurrent Firmware Updates not available for LC servers.
Concurrent system firmware update is not supported on these LC servers.
Use the following examples as a reference to determine whether your installation will be concurrent or disruptive.
For the LC server systems, the installation of system firmware is always disruptive.
The xxx.pnor file updates the primary side of the PNOR. The yyy.bin updates the primary side of the BMC only. The golden sides are unchanged.
Filename | Size | Checksum |
P9DSU20180531_IBM_prod_sign.pnor | 67108992 | 46cdbfee75ddf7f00c929e55aa187e4d |
SMT_P9_127.bin | 33554432 | 0a3a67defd11f6d4a1104ef2175fb5d8 |
PUpdate (218) | 133800 | a4ff602c68f125c9791c8ce834f23b25 |
Note: The Checksum can be found by running the Linux/Unix/AIX md5sum command against the file (all 32 characters of the checksum are listed), ie: md5sum pUpdate.
After a successful update to the new firmware level, the PNOR components and BMC should be at the following levels. The ipmitool "fru" command can be used to display FRU ID 47 and the ipmitool "mc info" command can be used to display the BMC level.
Note: FRU information for the PNOR level does not show the updated levels via the fru command until the system has been booted once at the updated level.
PNOR firmware levels from FRU ID 47 inventory list for driver:
Product Name : OpenPOWER Firmware
Product Version : open-power-SUPERMICRO-P9DSU-V1.16-20180531-prod
Product Extra : op-build-e4b3eb5
Product Extra : skiboot-v6.0-p1da203b
Product Extra : hostboot-f911e5c-pda8239f
Product Extra : occ-77bb5e6-p623d1cd
Product Extra : linux-4.16.7-openpower2-pbc45895
Product Extra : petitboot-v1.7.1-pf773c0d
Product Extra : machine-xml-218a77
BMC Level:
Display BMC firmware level using the "ipmitool mc info | grep Firmware" command:
Firmware Revision : 1.27
OP920.00 | |
V1.16-20180531/ OP920.02/BMC 1.27 10/15/2018
| Impact: Availability Severity: SPE
System firmware changes that affect all systems
A problem was fixed for a false over-temperature reading on the AOC SAS/SATA adapter due to an incorrect low 65C maximum temperature threshold. With the fix, the maximum operating temperature for the device is set to 100C as the upper threshold.
|
V1.16-20180531/ OP920.01/BMC 1.23 06/13/2018 | Impact: Availability Severity: SPE
System firmware changes that affect all systems
A problem was fixed for a failure in DDR4 RCD (Register Clock Driver) memory initialization that causes half of the DIMM memory to be unusable after an IPL. This is an intermittent problem where the memory can sometimes be recovered by doing another IPL. The error is not a hardware problem with the DIMM but it is an error in the initialization sequence needed get the DIMM ready for normal operations.
A problem was fixed for a failure to IPL if there are DIMM failures that require DIMM chips to be guarded. During the memory reconfiguration, the system would reboot itself before the reconfiguration could complete. The memory failure would persist into the next IPL attempt, with the result that the system might not be able to IPL with bad memory DIMMs. This problem is more prevalent when there are errors in the larger DIMM modules, such as the 128 GB chips, as it takes longer to reconfigure the ranks of the larger memory chips. There is a possibility that an IPL watchdog timeout and system reboot can occur if a rank of memory takes over two minutes to reconfigure to guard out the bad memory.
A problem was fixed for a failure to isolate to an errant FRU for a system checkstop. This is an intermittent error related to the On-Chip Controller (OCC) not waiting long enough to collect the failure information for a checkstop that occurs on a busy system. When this error happens, it prevents checkstop diagnosis procedures from identifying the cause of the checkstop fault. For this error, no active error bits are found and the checkstop analysis failure error log is mapped to a SEL which directs the customer to contact support as shown below:
1 | 05/17/2018 | 20:49:27 | System Firmware Progress Boot Progress | Motherboard initialization () | Asserted 2 | 05/17/2018 | 20:50:13 | System Firmware Progress Boot Progress | System boot initiated () | Asserted 3 | 05/18/2018 | 09:27:07 | OEM record c0 | 040020 | ceff6fffffff ==> Checkstop Signal , check other serviceable SELs and resolve them 4 | 05/18/2018 | 09:27:44 | System Firmware Progress Boot Progress | Motherboard initialization () | Asserted 5 | 05/18/2018 | 09:27:56 | OEM record df | 040020 | 12046faa0000 6 | 05/18/2018 | 09:28:05 | OEM record de | 000000 | 100000000005 ==> Procedure callout, decodes to contact next level of suppoort of assistance 7 | 05/18/2018 | 09:28:31 | System Firmware Progress Boot Progress | System boot initiated () | Asserted The following steps can be used to identify the signature for this problem. Look for the SEL indicating checkstop signal as shown below: 3 | 05/18/2018 | 09:27:07 | OEM record c0 | 040020 | ceff6fffffff ==> Checkstop Signal , check other serviceable SELs and resolve them If found, look for a "OEM de" SEL that is logged a few minutes after the checsktop signal SEL, decoding into "contact next level of support for assistance": 5 | 05/18/2018 | 09:27:56 | OEM record df | 040020 | 12046faa0000 6 | 05/18/2018 | 09:28:05 | OEM record de | 000000 | 100000000005 ==> Procedure callout, decodes to contact next level of suppoort of assistance If found, collect plc.pl output and inspect the decoded eSEL (PEL) associated with the checkstop analysis. If the signature description in the PRD log reads "No active error bits found", the checkstop analysis failure is confirmed. | Reference Code : BC70E550 | | Hex Words 2 - 5 : 000000E0 00000B00 00000000 00200000 | | Hex Words 6 - 9 : 000B0004 00000103 BC4ADD02 00000000 | | | | ModuleId : 0x0B | | Reason Code : 0xE550 | | Code Location : 0x0103 | | | | PRD SRC Type : PRD Detected Hardware Indication | | PRD SRC Class : Software likely caused a hardware error condition,| | : smaller possibility of a hardware cause. | | | | PRD Signature : 0x000B0004 0xBC4ADD02 | | Signature Description : mcs(n0p1c0) No active error bits found |
|
V1.16-20180518/ OP920.00 05/25/2018 | Impact: New Severity: New
New features and functions for MTMs 9006-12P and 9006-22P:
GA Level
Support for trusted boot and secure boot with TPM2.0 Nuvoton NPCT650ABAWX through an I2C controller.
Dedicated 1 GB IPMI port
Integrated MicroSemi PM8069 SAS/SATA 16-port Internal Storage Controller PCIe3.0 x8 with RAID 0, 1, 5, and 10 support (no write cache)
Integrated Intel XL710 Quad Port 10GBase-T PCIe3.0 x8 UIO built-in LAN (one shared management port)
Supermicro BMC support for Redfish: Get System/Chassis inventory info Manage user accounts and privileges BMC configuration (AD, LDAP, SNMP, SMTP, RADIUS, Fanmode, Mousemode, NTP, Snooping etc.) BIOS configuration Boot order change RAID configuration (For 3108) Storage Management Get NIC MAC info (NIC asset info) BMC/BIOS Firmware updates Get thermal/power/sensor info Get system memory info Get hostname Launch iKVM/HTML5 using redfish Update SSL certificate and key Perform computer system reset BMC reset BMC configuration reset Get health event log/Advanced system event log Acknowledge warning/critical severity events Virtual Media ISO image mounting HIPER/Pervasive: A firmware change was made to address a rare case where a memory correctable error on POWER9 servers may result in an undetected corruption of data.
|
OS levels supported by the LC921 and LC922 servers:
- Red Hat Enterprise Linux (RHEL) 7.5 little endian (LE) (POWER9), or later
- Ubuntu Server 18.04 LTS
- Virtualization Engine
Feature (#EC16) -OpenPOWER non-virtualized configuration (also known as bare-metal install)
IBM Power LC921 and LC922 servers support Linux which provides a UNIX like implementation across many computer architectures. Linux supports almost all of the Power System I/O and the configurator verifies support on order. For more information about the software that is available on IBM Power Systems, see the Linux on IBM Power Systems website:
http://www.ibm.com/systems/power/software/linux/index.html
The Linux operating system is an open source, cross-platform OS. It is supported on every Power Systems server IBM sells. Linux on Power Systems is the only Linux infrastructure that offers both scale-out and scale-up choices. One supported version of Linux on the IBM Power LC921 and LC922 servers is Ubuntu Server 18.04 LTS for IBM POWER9. For more information about Ubuntu Server for Ubuntu for POWER9 see the following website:
https://wiki.ubuntu.com/ppc64el
Another supported version of Linux on the LC921 and LC922 servers is Red Hat Enterprise Linux 7.5 LE. For additional questions about the availability of this release and supported Power servers, consult the Red Hat Hardware Catalog at https://access.redhat.com/products/red-hat-enterprise-linux/#addl-arch
For more information about Linux on Power, see the Linux on Power developer center at https://developer.ibm.com/linuxonpower/
For information about the features and external devices that are supported by Linux, see this website:
http://www.ibm.com/systems/power/software/linux/index.html
Use one of the following commands at the Linux command prompt to determine the current Linux level:
•cat /proc/version
•uname -a
The output string from the command will provide the Linux version level.
The opal-prd package on the Linux system collects the OPAL Processor Recovery Diagnostics messages to log file /var/log/syslog. It is recommended that this package be installed if it is not already present as it will help with maintaining the system processors by alerting the users to processor maintenance when needed.
On Red Hat Linux, perform command "rpm -qa | grep -i opal-prd ". The command output indicates the package is installed on your system if the rpm for opal-prd is found and displayed. This package provides a daemon to load and run the OpenPower firmware's Processor Recovery Diagnostics binary. This is responsible for run-time maintenance of Power hardware. If the package is not installed on your system, the following command can be run on Red Hat to install it:
sudo yum update opal-prd
The system firmware is a combination of BMC and PNOR firmware levels.
Use the ipmtool "fru" command or the BMC Web GUI FRU option to look at product details of FRU 47.
ipmitool -I lanplus -H <bmc host IP address> -P ADMIN -U ADMIN fru print 47
Use the ipmitool "mc info" command or the BMC Web GUI System tab to look at the BMC level.
ipmitool -I lanplus -H <bmc host IP address> -P ADMIN -U ADMIN mc info|grep "Firmware"
Follow the instructions on Fix Central. You must read and agree to the license agreement to obtain the firmware packages.
The updating and upgrading of system firmware depends on several factors, such as the current firmware that is installed, and what operating systems is running on the system.
These scenarios and the associated installation instructions are comprehensively outlined in the firmware section of Fix Central, found at the following website:
http://www.ibm.com/support/fixcentral/
Any hardware failures should be resolved before proceeding with the firmware updates to help insure the system will not be running degraded after the updates.
The pUpdate utility is provided with the firmware update files from IBM Fix Central. It can be used to perform in-band (from the host OS), in-band update recovery, and out-of-band updates by selecting either the "-i usb" , "-i bt" or "-i lan" parameters, respectively on the command invocation. The code update needs to be done in two steps: 1) Update the BMC firmware and 2) Update the CEC PNOR for the hostboot and the OPAL components. It is recommended that the BMC be updated first unless otherwise specified in the firmware install instructions.
Before using the pUpdate command on the host, make sure that the ipmi driver is loaded in the kernel and the ipmi service is started.
Note: For updates that use the "usb" or "bt" pUpdate option, you must use the root user ID and password to log in to the host operating system. After you log in to the host operating system, ensure that the IPMI service is activated.
# chkconfig ipmi on
# service ipmi start
For more information about activating the IPMI service, see the OpenIPMI Driver: https://www.ibm.com/support/knowledgecenter/POWER8/p8eih/p8eih_ipmi_open_driver.htm
For in-band update, use the following "-i usb" invocation of pUpdate:
BMC update: "pUpdate -f bmc.bin -i usb", where bmc.bin is the name and location of the BMC image file.
PNOR update: "pUpdate -pnor pnor.bin -i usb", where pnor.bin is the name and location of the PNOR image file.
If the in-band update fails on the BMC, use the recovery option with the Block Transfer (bt) invocation of pUpdate:
BMC update: "pUpdate -f bmc.bin -i bt " where bmc.bin is the name and location of the BMC image file.
PNOR update:" pUpdate -pnor pnor.bin -i bt " where pnor.bin is the name and location of the PNOR image file.
For more information on BMC recovery steps, refer to the following link in the IBM Knowledge Center:
https://www.ibm.com/support/knowledgecenter/POWER8/p8eis/p8eis_console_problem.htm
If the host is not booted, a network connection can be made to the BMC and an out-of-band update done with the following LAN invocation from a Linux companion system:
BMC update: " pUpdate -f bmc.bin -i lan -h xx.xx.xx.xx -u ADMIN -p ADMIN -r y" where bmc.bin is the name and location of the BMC image file, xx.xx.xx.xx is the IP address of the BMC.
PNOR update: "pUpdate -pnor pnor.bin -i lan -h xx.xx.xx.xx -u ADMIN -p ADMIN " where pnor.bin is the name and location of the PNOR image file and xx.xx.xx.xx is the IP address of the BMC.
For more details on how to use the pUpdate utility, refer to the following link:
https://www.ibm.com/support/knowledgecenter/POWER9/p9eit/p9eit_update_firmware_pupdate.htm
You can use diagnostic utilities to diagnose adapter problems.
For more details on how to use the diagnostic utilities, refer to the following link:
https://www.ibm.com/support/knowledgecenter/POWER9/p9eit/p9eit_diags_kickoff.htm
OpenPOWER requires Source Forge ipmitool level v1.8.15 to execute correctly on the P9 firmware.
Another method to update the system firmware is by using the baseboard management controller (BMC).
The system firmware is a combination of the BMC firmware and the PNOR firmware. To update the system firmware, update both the BMC firmware and the PNOR firmware by using the BMC.
Note: System firmware update from the BMC Web GUI is only supported on Google Chrome and Mozilla Firefox browsers.
Complete the following steps to update the BMC firmware:
1. Log in to the BMC by entering the user name and password. Then, press Enter.
2. From the Maintenance list on the BMC dashboard, select BMC Update.
3. In the BMC Update window, select Enter Update Mode. Click OK.
4. In the BMC Upload window, choose the .bin file from your local system folder and click Upload Firmware. Wait for the file to be uploaded. Then, click OK.
5. The existing and new versions of the BMC firmware are displayed. Ensure that the Preserve Configuration check box is selected and the Preserve SDR check box is not selected. Click Start Upgrade.
Note: You cannot perform other activities by using the BMC interface until the firmware update is complete.
6. The upgrade progress of the firmware update is displayed. After the BMC update is complete, the system is restarted.
7. After the restart of the system is complete, verify the firmware revision level in the System menu of the BMC dashboard.
Complete the following steps to update the PNOR firmware:
1. Log in to the BMC by entering the user name and password. Then, press Enter.
2. From the Maintenance list on the dashboard, select PNOR Update.
3. In the PNOR Upload window, choose the .pnor file from your local system folder and click Upload PNOR. Wait for the file to be uploaded. Then, click OK.
4. The existing and new dates of the PNOR firmware are displayed. Click Start Upgrade.
Note: You cannot perform other activities by using the BMC interface until the PNOR update is complete.
5. The progress of the PNOR update is displayed. After the PNOR update is completed, the system must be restarted to finish installation of the new PNOR firmware.
For more information on updating the firmware using the BMC, refer to the following link:
https://www.ibm.com/support/knowledgecenter/POWER9/p9eit/p9eit_update_firmware_bmc.htm
System I/O devices have firmware that can be updated.
Please see the IBM Knowledge Center for the 9006-12P and 9006-22P for applicable I/O firmware update information.
9006-12P:
http://www.ibm.com/support/knowledgecenter/POWER9/p9hdx/9006_12p_landing.htm
9006-22P:
http://www.ibm.com/support/knowledgecenter/POWER9/p9hdx/9006_22p_landing.htm
To check the firmware version on this device, use the Microsemi Adaptec cli utility arcconf . The command “arcconf getversion” will show information for all Microsemi Adaptec devices present on the machine. If no other Microsemi Adaptec slot cards are installed, the embedded controller will have the controller ID of “1”.
Part # = PM8069
Description: Embedded 12Gb SAS Controller, Microsemi Adaptec SmartIOC 2000 16i
Minimum FW level: 4.02[0] (0)
The service processor, or baseboard management controller (BMC), provides a hypervisor and operating system-independent layer that uses the robust error detection and self-healing functions that are built into the POWER processor and memory buffer modules. OpenPOWER application layer (OPAL) is the system firmware in the stack of POWER processor-based Linux-only servers.
The service processor, or baseboard management controller (BMC), is the primary control for autonomous sensor monitoring and event logging features on the LC server.
The BMC supports the Intelligent Platform Management Interface (IPMI) for system monitoring and management. The BMC monitors the operation of the firmware during the boot process and also monitors the OPAL hypervisor for termination. The firmware code update is supported through the BMC and Intelligent Platform Monitoring Interface (IPMI) and the BMC Web GUI The GUI console is accessed using a web browser with a "http:" connection to port. See section 1.2 for the supported browsers that can be used with BMC Web GUI.
The OpenPOWER Abstraction Layer (OPAL) provides hardware abstraction and run time services to the running host Operating System. For these LC servers, only the OPAL bare-metal installs can be used.
Find out more about OPAL skiboot here:
https://github.com/open-power/skiboot
The Intelligent Platform Management Interface (IPMI) is an open standard for monitoring, logging, recovery, inventory, and control of hardware that is implemented independent of the main CPU, BIOS, and OS. These LC servers provide one 10M/100M baseT IPMI port.
The ipmitool is a utility for managing and configuring devices that support IPMI. It provides a simple command-line interface to the service processor. You can install the ipmitool from the Linux distribution packages in your workstation, sourceforge.net, or another server (preferably on the same network as the installed server).
For installing ipmitool from sourceforge, please see section 1.1 "Minimum ipmitool Code Level".
For more information about ipmitool, there are several good references for ipmitool commands:
1.The man page
2.The built-in command line help provides a list of IPMItool commands:
# ipmitool help
3.You can also get help for many specific IPMItool commands by adding the word help after the command:
# ipmitool channel help
4.For a list of common ipmitool commands and help on each, you may use the following link:
www.ibm.com/support/knowledgecenter/linuxonibm/liabp/liabpcommonipmi.htm
To connect to your host system with IPMI, you need to know the IP address of the server and have
a valid password. To power on the server with the ipmitool, follow these steps:
1. Open a terminal program.
2. Power on your server with the ipmitool:
ipmitool -I lanplus -H bmc_ip_address -U ipmi_userid -P ipmi_password power on
3. Activate your IPMI console:
ipmitool -I lanplus -H bmc_ip_address -U ipmi_userid -P ipmi_password sol activate
Petitboot is a kexec based bootloader used by IBM POWER9 systems for doing the bare-metal installs on these LC servers.
After the POWER9 system powers on, the petitboot bootloader scans local boot devices and network interfaces to find boot options that are available to the system. Petitboot returns a list of boot options that are available to the system. If you are using a static IP or if you did not provide boot arguments in your network boot server, you must provide the details to petitboot. You can configure petitboot to find your boot with the following instructions:
https://www.ibm.com/support/knowledgecenter/linuxonibm/liabp/liabppetitbootadvanced.htm
You can edit petitboot configuration options, change the amount of time before Petitboot automatically boots, etc. with these instructions:
https://www.ibm.com/support/knowledgecenter/linuxonibm/liabp/liabppetitbootconfig.htm
After you select to boot the ISO media for the Linux distribution of your choice, the installer wizard for that Linux distribution walks you through the steps to set up disk options, your root password, time zones, and so on.
You can read more about the petitboot bootloader program here:
https://www.kernel.org/pub/linux/kernel/people/geoff/petitboot/petitboot.html
This guide helps you install Linux on a Power Systems server.
Overview
Use the information found in http://www.ibm.com/support/knowledgecenter/linuxonibm/liabw/liabwkickoff.htm to install Linux on a non-virtualized (bare metal) IBM Power LC server. Note that the choice of PowerKVM is offered in the link but that is not a supported OS for these LC servers.
Date | Description |
10/15/2018 | OP920.02 release (reverted pUpdate tool to prior version, 218) |
6/13/2018 | OP920.01 release |
05/25/2018 | OP920.00 release for Power 9 LC921 (9006-12P) and LC922(9006-22P) |
|
|