IBM HMC on Power Firmware

Applies to:  7063-CR2

This document provides information about the installation of Licensed Machine or Licensed Internal Code, which is sometimes referred to generically as microcode or firmware.

Contents

1.0 Systems Affected

This package provides firmware for IBM HMC on Power 7063-CR2 only with minimum HMC version level of  HMC V9 R2 M950 or later.

The firmware level in this package is:

• •PNOR OP9_v2.5_4.115 and OpenBMC op940.hmc-5 

1.1 Minimum ipmitool Code Level

This section specifies the "Minimum ipmitool Code Level" required by the System Firmware for managing the system.   OpenPOWER requires ipmitool level v1.8.15 or later to execute correctly on the OP910  and later firmware.  It must be capable of establishing a IPMI v2 session with the ipmi support on the BMC.

 Verify your ipmitool level on your Linux workstation using the following command:

bash-4.1$ ipmitool -V

ipmitool version 1.8.15

If you are need to update or add impitool to your Linux workstation , you can compile ipmitools (current level 1.8.15) for Linux as follows from Sourceforge:

1.1.1  Download impitool tar from http://sourceforge.net/projects/ipmitool/  to  your linux system

1.1.2  Extract tarball on Linux system

1.1.3  cd to top-level directory

1.1.4 ./configure

1.1.5  make

1.1.6  ipmitool will be under src/ipmitool        

You may also get the ipmitool package directly from your workstation Linux packages.

1.2 Fix level Information on IBM Power HMC Components and Operating systems

For specific fix level information on key components of IBM HMC model 7063-CR2 and the HMC software, please refer to the documentation in the IBM Knowledge Center.

https://www.ibm.com/docs/en/power9/0000-FUL?topic=code-upgrading-your-hmc-software

2.0 Important Information

Downgrading firmware from any given release level to an earlier release level is not recommended.

If you feel that it is necessary to downgrade the firmware on your system to an earlier release level, please contact your next level of support.

Concurrent Firmware Updates not available for Power HMC.

Concurrent system firmware update is not supported on the Power HMC.

3.0 Firmware Information

Use the following examples as a reference to determine whether your installation will be concurrent or disruptive.

For the Power HMC, the installation of system firmware is always disruptive.

3.1 Firmware Information and Description

The BMC and PNOR image tar files are used to update the primary side of the PNOR and the primary side of the BMC only, leaving the golden sides unchanged.

Note: The Checksum can be found by running the Linux/Unix/AIX md5sum command against the Hardware Platform Management (hpm) file (all 32 characters of the checksum are listed), ie: md5sum <filename>

After a successful update to this firmware level, the PNOR components and BMC should be at the following levels.  

To display the PNOR level, use the following BMC command:  "cat /var/lib/phosphor-software-manager/pnor/ro/VERSION | grep -A 12 IBM"

And the BMC command line command "cat" can be used to display the BMC level:  "cat /etc/os-release".

Note:  FRU information for the PNOR level does not show the updated levels via the fru command until the system has been booted once at the updated level.

PNOR firmware level:         driver content

  IBM-mowgli-OP9_v2.5_4.115-prod

    op-build-v2.3-rc2-1412-g22463a8

    buildroot-2019.05.3-20-g4a064c9

    skiboot-v6.7

    hostboot-0896112-p067a861

    occ-9047e57

    linux-5.4.107-openpower1-p2d66075

    petitboot-v1.12

    machine-xml-59f3878

    hostboot-binaries-hw101520a.op940

    capp-ucode-p9-dd2-v4

    sbe-47abe2a-p72ee8b7

    hcode-hw031221a.op940

BMC firmware level:         driver content

BMC Primary side version:

 op940.hmc-5

Alternatively access the OpenBMC UI by directly pointing a browser to the IP or hostname of the BMC:

Example: https://<ip or hostname>

In the “Server overview” panel, locate the “FIRMWARE VERSION” fields under “Server information”(PNOR), and “BMC information (BMC).

4.0 Operating System Information

OS levels supported by the 7063-CR2 server:

- HMC V9 R2 M950 or later

4.1 HMC on Power Operating System

The HMC stack runs on an embedded Linux distribution. The HMC on Power version V9 R2 M950 or later is supported on the 7063-CR2.

4.2 How to Determine the Level of the HMC

Use the following steps in the below link to navigate the HMC GUI to determine the HMC level:

https://www.ibm.com/docs/en/power9/9009-42A?topic=code-determining-your-hmc-machine-version-release

From the HMC command line use “lshmc -V”.

https://www.ibm.com/support/knowledgecenter/8247-21L/p8hai/p8hai_viewcodelevel_enh.htm

5.0 How to Determine The Currently Installed Firmware Level

See Section 3.1 “Firmware Information and Description”.

6.0 Downloading the Firmware Package

Follow the instructions on Fix Central. You must read and agree to the license agreement to obtain the firmware packages.

7.0 Installing the Firmware

The updating and upgrading of system firmware depends on several factors, such as the current firmware that is installed, and what operating systems is running on the system.

These scenarios and the associated installation instructions are comprehensively outlined in the firmware section of Fix Central, found at the following website:

http://www.ibm.com/support/fixcentral/

Any hardware failures should be resolved before proceeding with the firmware updates to help insure the system will not be running degraded after the updates.

The 7063-CR2 firmware is made up of two separate components

• •.BMC - image for the Baseboard Management Controller 

• •.Server image - PNOR 

A boot priority system, allows for the selection of a previous image to be used. This is useful when there is a need to revert to a prior image.

On the OpenBMC UI, the image file that is listed at the top (for each stack, BMC and PNOR), the image with the highest boot priority, is used the next time that the device is booted. You can change the boot order for the image file by clicking the arrow icons.

Image State Definitions

• •.Functional: The running image on the device. 

• •.Active: The image is available to boot from, but is not currently the running image. If the image is the top image in the relevant table, it becomes the functional image the next time the device is rebooted. 

• •.Activating: The image is in the process of being activated and becomes either Active or Failed. 

• •.Failed: The image failed to activate. 

• •.Ready: The image is ready to be activated. 

• •.Invalid: This image is an invalid image and cannot be activated. 

The OpenBMC UI or the openbmctool commands can be used to both view or update the firmware images.

The HMC must be shut down prior to updating firmware.

It is recommended that both BMC and PNOR be updated prior to restarting the HMC.

7.1 Updating Firmware Using OpenBMC GUI

7.1.1 Update the BMC image

1. Access Server configuration -> Firmware

2. Scroll down on the page to locate the section to upload the firmware image.

3. Click on Choose a file and browse your local filesystem for the location of the BMC image.

4. Click on Upload firmware

The “Upload in progress...” message is displayed  

A confirmation message is displayed when the upload is complete.

5. The new image is now in a Ready state. Click Activate, under the Action column to activate it.

6. The user is presented with a confirmation panel with the options to "ACTIVATE FIRMWARE FILE WITHOUT REBOOTING BMC" and "ACTIVATE FIRMWARE FILE AND AUTOMATICALLY REBOOT BMC".

7. Select ACTIVATE FIRMWARE FILE AND AUTOMATICALLY REBOOT BMC and click Continue

8. The image state will show Activating

9. The image will then transition to Functional state (the running image on the device)

10.  If successful, a message is displayed on the upper right of the session, showing Success! BMC is rebooting.

11  After the BMC comes back up and the UI is refreshed, the new image is now listed on the top line (first in boot priority) and the previous image is now listed second. The original second image was removed.

12.  This concludes the BMC part of the update.

7.1.2 Update the PNOR image

The next firmware image to update is the PNOR (also known as Server) image.

1. Scroll down on the page to locate the section to upload the firmware image.

2. Click on Choose a file and browse your local filesystem for the location of the PNOR image.

3. Click on Upload firmware

The “Upload in progress... “message is displayed

 A confirmation message is displayed when the upload is complete:

4. The new image is now in a Ready state. Click Activate, under the Action column to activate it.

5. The user is presented with a confirmation panel  “Confirm server firmware file activation”.with the options to "ACTIVATE FIRMWARE FILE WITHOUT BOOTING SERVER" and "ACTIVATE FIRMWARE FILE AND AUTOMATICALLY BOOT SERVER".

6. Select ACTIVATE FIRMWARE FILE AND AUTOMATICALLY BOOT SERVER and click Continue

7. The image state will show Activating

8. The image will then transition to Functional state (the running image on the device)

9. After a UI refresh, the new image is now listed on the top line (first in boot priority) and the previous image is now listed second. Any original second image is removed (there wasn't one in this example).

10. This concludes the PNOR part of the update.

11. The system is automatically started following PNOR update.

7.2 Updating Firmware Using openbmctool

The process of updating firmware on the OpenBMC Power HMC is documented below.

The sequence of events that must happen is the following:

•Power off the Host

•Update and Activate BMC

•Update and Activate PNOR

•Reboot the BMC (applies new BMC image)

•Power on the Host (applies new PNOR image)

The OpenBMC firmware updates (BMC and PNOR)  for the Power HMC can be managed via the command line with the openbmctool.

The openbmctool is obtained using the IBM Support Portal.

1. 1.Go to the IBM Support Portal. 

2. 2.In the search field, enter your machine type and model.  Then click the correct product support entry for your system. 

3. 3.From the Downloads list, click the openbmctool for your machine type and model. 

4. 4.Follow the instructions to install and run the openbmctool.  You will need to provide the file locations of the BMC firmware image tar and PNOR firmware image tar that must be downloaded from Fix Central for the update level needed. 

Information on the openbmctool and the firmware update process can be found in the IBM Knowledge Center:  

https://www.ibm.com/support/knowledgecenter/POWER9/p9ei8/p9ei8_update_firmware_openbmctool.htm

8.0 System Management and Virtualization

The service processor, or baseboard management controller (BMC), provides a hypervisor and operating system-independent layer that uses the robust error detection and self-healing functions that are built into the POWER processor and memory buffer modules. OpenPOWER application layer (OPAL) is the system firmware in the stack of POWER processor-based Linux-only servers.

8.1  BMC Service Processor IPMI

The service processor, or baseboard management controller (BMC), is the primary control for autonomous sensor monitoring and event logging features on the Power HMC.

The BMC supports the Intelligent Platform Management Interface (IPMI) for system monitoring and management.  The BMC monitors the operation of the firmware during the boot process and also monitors the OPAL hypervisor for termination.

8.1.1  Security Risks Associated with Using IPMI

Various risks that are associated with the Intelligent Platform Management Interface (IPMI) have been identified and documented in the information technology (IT) security community.

Possible risks includes the following three common vulnerabilities and exposures (CVEs):

1) CVE-2013-4037:

The Remote Authenticated Key-Exchange Protocol (RAKP), which is specified by the IPMI standard for authentication, has flaws. Although the system does not allow the use of null passwords, a hacker might reverse engineer the RAKP transactions to determine a password. The authentication process for IPMI requires the management controller to send a hash of the requested password of the user to the client before the client authenticates. This process is a key part of the IPMI specification. The password hash can be broken by using an offline brute force or dictionary attack.

2) CVE-2013-4031:

IBM Power Systems and OpenPower Systems are preconfigured with one IPMI user account, which has the same default login name and password on all affected systems. If a malicious user gains access to the IPMI interface by using this preconfigured account, the user can power off or on, or restart the host server, and create or change user accounts possibly preventing legitimate users from accessing the system. On OpenPower Systems, the default IPMI user name is root.  Additionally, if a user fails to change the default user name and password on each of the systems that is deployed, the user has the same login information for each of those systems.

3) CVE-2013-4786:

The IPMI 2.0 specification supports RMCP+ Authenticated Key-Exchange Protocol (RAKP) authentication, which allows remote attackers to obtain password hashes and conduct offline password guessing attacks by obtaining the hash-based message authentication code (HMAC) from a RAKP message 2 response from a BMC.

If a user is not managing a server by using the IPMI, one can configure the system to disallow IPMI network access from the user accounts. This task can be accomplished by using the IPMItool utility or a similar utility for managing and configuring the IPMI management controllers.  Use the following IPMItool command to disable the network access for an IPMI user:

ipmitool channel setaccess 1 #user_slot# privilege=15

For more information on the IPMI security vulnerabilities and configuration options and best practices to minimize the risks of this interface, go to the IBM Knowledge Center at the following URL:

https://www.ibm.com/support/knowledgecenter/POWER9/p9eih/p9eih_openbmc_security.htm

8.2 OpenPOWER Abstraction Layer (OPAL)

The OpenPOWER Abstraction Layer (OPAL) provides hardware abstraction and run time services to the running host Operating System.

For the 7063-CR2 server, only the OPAL bare-metal installs of the Hardware Management Console are supported.

Find out more about OPAL skiboot here:

https://github.com/open-power/skiboot

8.3 Petitboot bootloader

Petitboot is a kexec based bootloader used by IBM POWER9 systems for doing the bare-metal installs.

After the POWER system powers on, the petitboot bootloader scans local boot devices and network interfaces to find boot options that are available to the system. Petitboot returns a list of boot options that are available to the system. If you are using a static IP or if you did not provide boot arguments in your network boot server, you must provide the details to petitboot. You can configure petitboot to find your boot with the following instructions:

https://www.ibm.com/support/knowledgecenter/linuxonibm/liabw/liabppetitbootadvanced.html

You can edit petitboot configuration options, change the amount of time before Petitboot automatically boots, etc. with these instructions:

https://www.ibm.com/support/knowledgecenter/linuxonibm/liabw/liabppetitbootconfig.html

After you select to boot the ISO media for the Linux distribution of your choice, the installer wizard for that Linux distribution walks you through the steps to set up disk options, your root password, time zones, and so on.

You can read more about the petitboot bootloader program here:

https://www.kernel.org/pub/linux/kernel/people/geoff/petitboot/petitboot.html

9.0 Change History