SC860_103_056 / FW860.30
06/30/17 |
Impact: Availability
Severity: SPE
New features and functions
- Support was added for Redfish API to allow the ISO 8610
extended format for the time and date so that the date/time can be
represented as an offset from UTC (Universal Coordinated Time).
- Support for the Redfish API for power and thermal
properties for the chassis. The new URIs are as follows::
https://<fsp ip>/redfish/v1/Chassis/<id>/Power :
Provides fan data
https://<fsp ip>/redfish/v1/Chassis/<id>/Thermal : Provides
power supply data
Only the Redfish GET operation is supported for these resources.
System firmware changes that affect all systems
- A problem was fixed
for service actions with SRC B150F138 missing an Advanced System
Management Interface (ASMI) Deconfiguration Record. The
deconfiguration records make it easier to organize the repairs that are
needed for the system and they need to be consistent with the periodic
maintenance reminders that are logged for the failed FRUs.
- A problem was fixed for a false 1100026B1 (12V power good
failure) caused by an I2C bus write error for a LED state. This
error can be triggered by the fan LEDs changing state.
- A problem was fixed for a fan LED turning amber on solid
when there is no fan fault, or when the fan fault is for a different
fan. This error can be triggered anytime a fan LED needs to
change its state. The fan LEDs can be recovered to a normal state
concurrently using the following link steps for a soft reset of the
service processor: https://www.ibm.com/support/knowledgecenter/POWER8/p8hby/p8hby_softreset.htm
- A problem was fixed for sporadic blinking amber LEDs for
the system fans with no SRCs logged. There was no problem with
the fans. The LED corruption occurred when two service processor
tasks attempted to update the LED state at the same time. The fan
LEDs can be recovered to a normal state concurrently using the
following link steps for a soft reset of the service processor: https://www.ibm.com/support/knowledgecenter/POWER8/p8hby/p8hby_softreset.htm
- A problem was fixed for a Redfish Patch on the "Chassis" or
"IBMEnterpriseComputerSystem" with empty data that caused a "500
Internal Server Error". Validation for the empty data case has
been added to prevent the server error.
- A problem was fixed for hardware dumps only collecting data
for the master processor if a run-time service processor failover had
occurred prior to the dump. Therefore, there would be only master
chip and master core data in the event of a core unit checkstop.
To recover to a system state that is able to do a full collection of
debug data for all processors and cores after a run-time failover, a
re-IPL of the system is needed.
- A problem was fixed for a Redfish Patch on power mode to
"MaxPowerSaver" that caused a "500 Internal Server Error" when
that power mode was not supported on the system. With the fix,
the Redfish server response is a list of the valid power modes that be
used for the system.
- A problem was fixed for the loss of Operations Panel
function 30 (displaying ethernet port HMC1 and HMC2 IP addresses)
after a concurrent repair of the Operations Panel.
Operations Panel function 30 can be restored concurrently using
the following link steps for a soft reset of the service
processor: https://www.ibm.com/support/knowledgecenter/POWER8/p8hby/p8hby_softreset.htm
- A problem was fixed for a core dump of the rtiminit
(service processor time of day) process that logs an SRC B15A3303
and could invalidate the time on the service processor. If the
error occurs while the system is powered on, the hypervisor has the
master time and will refresh the service processor time, so no action
is needed for recovery. If the error occurs while the system is
powered off, the service processor time must be corrected on the
systems having only a single service processor. Use the following
steps from the IBM Knowledge Center to change the UTC time with the
Advanced System Management Interface: https://www.ibm.com/support/knowledgecenter/en/POWER8/p8hby/viewtime.htm.
- A problem was fixed for the service processor boot
watch-dog timer expiring too soon during DRAM initialization in the
reset/reload, causing the service processor to go unresponsive.
On systems with a single service processor, the SRC B1817212 was
displayed on the control panel. For systems with redundant
service processors, the failing service processor was
deconfigured. To recover the failed service processor, the system
will need to be powered off with AC powered removed during a regularly
scheduled system service action. This problem is intermittent and
very infrequent as most of the reset/reloads of the service processor
will work correctly to restore the service processor to a normal
operating state.
- A problem was fixed for host-initiated resets of the
service processor causing the system to terminate. A prior fix
for this problem did not work correctly because some of the
host-initiated resets were being translated to unknown reset types that
caused the system to terminate. With this new correction for
failed host-initiated resets, the service processor will still be
unresponsive but the system and partitions will continue to run.
On systems with a single service processor, the SRC B1817212 will be
displayed on the control panel. For systems with redundant
service processors, the failing service processor will be
deconfigured. To recover the failed service processor, the system
will need to be powered off with AC powered removed during a regularly
scheduled system service action. This problem is intermittent and
very infrequent as most of the host-initiated resets of the service
processor will work correctly to restore the service processor to a
normal operating state.
- A problem was fixed for a service processor reset triggered
by a spurious false IIC interrupt request in the kernel. On
systems with a single service processor, the SRC B1817201 is displayed
on the Operator Panel. For systems with redundant service
processors, an error failover to the backup service processor
occurs. The problem is extremely infrequent and does not impact
processes on the running system.
- A problem was fixed for the System Attention LED failing to
light for an error failover for the redundant service processors with
an SRC B1812028 logged.
- A problem was fixed for a system failure at run time with
SRC B111E450 corefir(55) that could not reIPL. A system node
should have been deconfigured for an ABUS error on a processor chip but
instead, the system was terminated. To recover from this problem,
manually guard the node containing the failed processor and then the
IPL will be successful.
- A problem was fixed for an incorrect Redfish error message
when trying to use the $metadata URI: "The resource at the
URI https://<systemip>/redfish/v1/%24metadata was not found.".
This %24 is meaningless. The "%24" has been replaced with a "$"
in the error message. The Redfish $metadata URI is not supported.
- A problem was fixed for a system failure caused by Host
boot problems with one node but the other nodes good. With the
fix, the node that is failing the Hostboot is deconfigured and the
system is able to IPL on the remaining nodes. To recover from
this problem, manually guard the node that is failing and reIPL.
System firmware changes that affect certain systems
- DEFERRED: On
systems using PowerVM firmware, a problem was fixed for PCIe3 I/O
expansion drawer (#EMX0) link improved stability. The settings
for the continuous time linear equalizers (CTLE) was updated for all
the PCIe adapters for the PCIe links to the expansion drawer. The
system must be re-IPLed for the fix to activate.
- On systems using
PowerVM firmware with a Linux Little Endian (LE) partition, a problem
was fixed for system reset interrupts returning the wrong values in the
debug output for the NIP and MSR registers. This problem reduces
the ability to debug hung Linux partitions using system reset
interrupts. The error occurs every time a system reset interrupt
is used on a Linux LE partition.
- On systems using PowerVM firmware, a problem was fixed for
"Time Power On" enabled partitions not being capable of suspend and
resume operations. This means Live Partition Mobility (LPM) would
not be able to migrate this type of partition. As a workaround,
the partition could be transitioned to a "Non-time Power On" state and
then made capable of suspend and resume operations.
- On systems using PowerVM firmware, a problem was fixed for
manual vNIC failovers (from the HMC, manually "Make the Backing Device
Active") so that the selected server was chosen for the failover,
regardless of its priority. With the problem, the server chosen
for the VNIC failover will be the one with the most favorable
priority.
There are two possible workarounds to the problem:
(1) Disable auto-priority-failover; Change priority to the server that
is needed as the target of the failover; Force the vNIC failover;
Change priority back to original setting.
(2) Or use auto-priority-failover and change the priority so the server
that is needed as the target of the failover is favored.
- On systems using PowerVM firmware, a problem was fixed for
extra error logs in the VIOS due to failovers taking place while the
client vNIC is inactive. The inactive client vNIC failovers are
skipped unless the force flag is on. With the problem occurring,
Enhanced Error Handling (EEH) Freeze/Temporary Error/Recovery logs
posted in the VIOS error log of the client partition boot can be
ignored unless an actual problem is experienced.
- On systems using PowerVM firmware, a problem was fixed for
a Live Partition Mobility (LPM) migration abort and reboot on the
FW860 target CEC caused by a mismatched address space for the
source and target partition. The occurrence of this problem is
very rare and related to performance improvements made in the memory
management on the FW860 system that exposed a timing window in the
partition memory validation for the migration. The reboot of the
migrated partition recovers from the problem as the migration was
otherwise successful.
- On systems using PowerVM firmware, a problem was fixed for
reboot retries for IBM i partitions such that the first load source I/O
adapter (IOA) is retried instead of bypassed after the first failed
attempt. The reboot retries are done for an hour before the
reboot process gives up. This error can occur if there is more
than one known load source, and the IOA of the first load source is
different from the IOA of the last load source. The error can be
circumvented by retrying the boot of the partition after the load
source device has become available.
- On systems using PowerVM firmware, a problem was fixed for
adapters failing to transition to shared SR-IOV mode on the IPL after
changing the adapter from dedicated mode. This intermittent
problem could occur on systems using SR-IOV with very large memory
configurations.
- On systems using PowerVM firmware, a problem
was fixed for SR-IOV adapters in shared mode for a transmission stall
or time out with SRC B400FF01 logged. The time out happens during
Virtual Function (VF) shutdowns and during Function Level Resets (FLRs)
with network traffic running.
This fix updates adapter firmware to 10.2.252.1927, for the following
Feature Codes: EN15, EN16, EN17, EN18, EN0H, EN0J, EN0M, EN0N, EN0K,
EN0L, EL38, EL3C, EL56, and EL57.
The SR-IOV adapter firmware level update for the shared-mode adapters
happens under user control to prevent unexpected temporary outages on
the adapters. A system reboot will update all SR-IOV shared-mode
adapters with the new firmware level. In addition, when an
adapter is first set to SR-IOV shared mode, the adapter firmware is
updated to the latest level available with the system firmware (and it
is also updated automatically during maintenance operations, such as
when the adapter is stopped or replaced). And lastly, selective
manual updates of the SR-IOV adapters can be performed using the
Hardware Management Console (HMC). To selectively update the
adapter firmware, follow the steps given at the IBM Knowledge Center
for using HMC to make the updates: https://www.ibm.com/support/knowledgecenter/HW4M4/p8efd/p8efd_updating_sriov_firmware.htm.
Note: Adapters that are capable of running in SR-IOV mode, but are
currently running in dedicated mode and assigned to a partition, can be
updated concurrently either by the OS that owns the adapter or the
managing HMC (if OS is AIX or VIOS and RMC is running).
- On systems with maximum memory configurations (where every
DIMM slot is populated - size of DIMM does not matter), a problem
has been fixed for systems losing performance and going into Safe mode
(a power mode with reduced processor frequencies intended to protect
the system from overheating and excessive power consumption) with
B1xx2AC3/B1xx2AC4 SRCs logged. This happened because of On-Chip
Controller (OCC) timeout errors when collecting Analog Power Subsystem
Sweep (APSS) data, used by the OCC to tune the processor
frequency. This problem occurs more frequently on systems that
are running heavy workloads. Recovery from Safe mode back to
normal performance can be done with a re-IPL of the system, or
concurrently using the following link steps for a soft reset of the
service processor: https://www.ibm.com/support/knowledgecenter/POWER8/p8hby/p8hby_softreset.htm.
To check or validate that Safe mode is not active on the system will
require a dynamic celogin password from IBM Support to use the service
processor command line:
1) Log into ASMI as celogin with dynamic celogin password
generated by IBM Support
2) Select System Service Aids
3) Select Service Processor Command Line
4) Enter "tmgtclient --query_mode_and_function" from the command line
The first line of the output, "currSysPwrMode" should say "NOMINAL" and
this means the system is in normal mode and that Safe mode is not
active.
- A problem has been fixed for systems losing
performance and going into Safe mode (a power mode with reduced
processor frequencies intended to protect the system from overheating
and excessive power consumption) with B1xx2AC3/B1xx2AC4 SRCs
logged. This happened because of an On-Chip Controller (OCC)
internal queue overflow. The problem has only been observed for systems
running heavy workloads with maximum memory configurations (where every
DIMM slot is populated - size of DIMM does not matter), but this may
not be required to encounter the problem. Recovery from Safe mode
back to normal performance can be done with a re-IPL of the system, or
concurrently using the following link steps for a soft reset of the
service processor: https://www.ibm.com/support/knowledgecenter/POWER8/p8hby/p8hby_softreset.htm.
To check or validate that Safe mode is not active on the system will
require a dynamic celogin password from IBM Support to use the service
processor command line:
1) Log into ASMI as celogin with dynamic celogin password
generated by IBM Support
2) Select System Service Aids
3) Select Service Processor Command Line
4) Enter "tmgtclient --query_mode_and_function" from the command line
The first line of the output, "currSysPwrMode" should say "NOMINAL" and
this means the system is in normal mode and that Safe mode is not
active.
- On systems using PowerVM firmware, a problem
was fixed for a partition boot from a USB 3.0 device that has an error
log SRC BA210003. The error is triggered by an Open Firmware
entry to the trace buffer during the partition boot. The error
log can be ignored as the boot is successful to the OS.
- On systems using PowerVM firmware, a problem
was fixed for a partition boot fail or hang from a Fibre Channel device
having fabric faults. Some of the fabric errors returned by the
VIOS are not interpreted correctly by the Open Firmware VFC drive,
causing the hang instead of generating helpful error logs.
- On systems with redundant service processors, a
problem was fixed for an extra SRC B150F138 logged for a power supply
that had already been replaced. The problem was triggered by a
service processor failover and an old power supply fault event that was
not cleared on the backup service processor. This caused the SRC
B150F138 to be logged for a second time. This problem can be
circumvented by clearing the error log associated with the bad FRU when
the FRU is replaced.
- On systems using PowerVM firmware, a problem was fixed for
a Power Enterprise Pool (PEP) resource Grace Period not being reset
when the server is in the "Out of Compliance" state and the resource
has been returned to put the server back in Compliance. The Grace
Period was not being reset after a double-commit of a resource (doing
an "remove" of an active resource) was resolved by restarting the
server with the double-committed resource. When Grace Period ends, the
"double-committed" resources on the server have to have been freed up
from use to prevent the server from going to "Out of Compliance".
If the user fails to free up the resource, the PEP is in an "Out of
Compliance" state, and the only PEP actions allowed are ones to free up
the double-commit. Once that is completed, the PEP is back In
Compliance. The loss of the Grace Period for the error makes it
difficult to move resources around in the PEP. Without the fix,
the user can "Add" another PEP resource to the server, and the
action of adding a PEP resource resets the Grace Period timer.
One could then "Remove" that one PEP resource just added, and then any
further "removes" of PEP resources would behave as expected with the
full Grace Period in effect.
- On systems using PowerVM firmware, a problem was
fixed for Power Enterprise Pool (PEP) IFL processors assignments
causing an "Out of Compliance" for normal processor licenses. The
number of IFL processors purchased was first credited as satisfying any
"unreturned" PEP processor resources, thus potentially leaving the
system "Out Of Compliance" since IFL processors should not be taking
the place of the normal (expensive) processor usage. In this
situation, without the fix, the user will need to either purchase more
"expensive" non-IFL processors to satisfy the non-IFL workloads or
adjust the partitions to reduce the usage of non-IFL processors.
This is a very infrequent problem for the following reasons:
1) PEP processors are infrequently left "unreturned" for short periods
of time for specialized operations such as LPM migrations
2) The user would have to purchase IFL processors from IBM, which is
not a common occurrence.
3) The user would have to put in a COD key for IFL processors while a
PEP processor is still "unreturned"
- On systems using PowerVM firmware, a problem
was fixed for a power off hanging at D200C1FF caused by a vNIC VF
failover error with SRC B200F011. The power off hang error is
infrequent because it requires that a VF failover error having occurred
first. The system can be recovered by using the power off
immediate option from the Hardware Management Console (HMC).
- On systems using PowerVM firmware, a problem was fixed for
the incorrect reporting of the Universally Unique Identifier (UUID) to
the OS, which prevented the tracking of a partition as it moved within
a data center. The UUID value as seen on HMC or the NovaLink did
not match the value as displayed in the OS.
- On systems using PowerVM firmware, a problem was fixed for
an error finding the partition load source that has a GPT format.
GUID Partition Table (GPT) is a standard for the layout of the
partition table on a physical storage device used in the server, such
as a hard disk drive or solid-state drive, using globally unique
identifiers (GUID). Other drives that are working may be using
the older master boot record (MBR) partition table format. This
problem occurs whenever load sources utilizing the GPT format occur in
other than the first entry of the boot table. Without the fix, a
GPT disk drive must be the first entry in the boot table to be able to
use it to boot a partition.
- On systems using PowerVM firmware, a problem was fixed for
an SRC BA090006 serviceable event log occurring whenever an attempt was
made to boot from an ALUA (Asymmetric Logical Unit Access)
drive. These drives are always busy by design and cannot be used
for a partition boot, but no service action is required if a user
inadvertently tries to do that. Therefore, the SRC was changed to
be an informational log.
|