Power7 High-End System Firmware

11/02/12

System firmware changes that affect all systems

• DEFERRED:  A problem was fixed that could cause a live lock on the power bus resulting in a system crash.
  
• The firmware was enhanced to increase the performance of certain applications by updating the routing tables.
• A problem was fixed that caused a segmentation fault in the service processor firmware.  When this occurred, a PERC error with SRC B181C350 was logged.
• On systems on which Internet Explorer (IE) is used to access the Advanced System Management Interface (ASMI) on the Hardware Management Console (HMC), a problem was fixed that caused IE to hang for about 10 minutes after saving changes to network parameters on the ASMI.
• A problem was fixed that caused the gateway network address  to be shown incorrectly on the System Management Services (SMS) menus when booting a partition on an iSCSI network.
• A problem was fixed that caused a "code accept" during a concurrent firmware installation from the HMC to fail with SRC E302F85C.
• On storage drawers in a cross-coupled topology, an attempt to place an indirect (failover) route at an SNID location in the SRT1 route table may result in a failover route that uses the opposite compute sub-cluster as a bounce point.  The firmware was enhanced to prevent this, since there are no physical links between the two compute sub-clusters in a cross-coupled topology.  Having a failover route through the opposite compute sub-cluster will lead to packet loss and application failure.
  
• A problem was fixed that prevented predictive guard errors from being deleted on the secondary service processor.  This caused hardware to be erroneously guarded out if a service processor failover occurred.
• A problem was fixed that caused the service processor to be reset during a CEC power off or reboot.  This causes the system to terminate, followed by a platform reboot.  SRC B181E6C7 is typically logged when this problem occurs.
• A problem was fixed that caused a system crash with unrecoverable SRC B7000103 and "ErFlightRecorder" in the failing stack.
• A problem was fixed that caused the following symptoms on user-level jobs:
  
    1.  During job initialization when starting communication over the cluster fabric, an error message similar to the following:
            4:ERROR 629 fD4fs: Message type 21 from source 4 4:MPI-PAMI ERROR: pami_init() failed with rc(1) 4:ERROR: 0031-309 Connect failed during message
             passing initialization, task 4, reason:
     2. The initialization may succeed, but an HFI translation failure may occur, causing a time out on the cluster network and other side effects.
  

• A problem was fixed that caused the dual-port Ethernet adapter, F/C 5270 and F/C 5708, to fail to power on with SRC B7006970.
• On systems in a high-performance computing (HPC) cluster in 8D topology, a problem was fixed that caused a secondary route to be linked to an indirect route chain.  Jobs that are run in indirect route mode may experience hangs and performance problems.
  
• The firmware was enhanced to improve the performance when indirect routing is used in large cluster systems.
  

06/27/12

System firmware changes that affect all systems

• A problem was fixed that caused a segmentation fault in the service processor firmware.  When this occurred, a perc error with SRC B181C350 was logged.
  

• On nodes with a single DCCA running AS730_093, a problem was fixed that prevented the node from booting, with SRC 10008732 erroneously logged.
  

06/13/12

System firmware changes that affect all systems

• DEFERRED:  The firmware was enhanced to fix a potential performance degradation on systems utilizing the stride-N stream prefetch instructions dcbt (with TH=1011) or dcbtst (with TH=1011).  Typical applications executing these algorithms include High Performance Computing, data intensive applications exploiting streaming instruction prefetchs, and applications utilizing the Engineering and Scientific Subroutine Library (ESSL) 5.1.
• The firmware was enhanced to correctly handle bus errors between the P7 processor chip and the I/O hub chip.
• The firmware was enhanced to correctly diagnose the failing FRU when SRC B1xxE504 with error signature "MCFIR[14] - Hang timer detector" was logged.
• The firmware was enhanced to improve the FRU callouts when the number of multi-bit errors on a POWER7 processor bus exceeds the threshold.  This reduces the number of FRUs replaced on a failing system.
• A problem was fixed the caused a system to crash when the system was in low power (or safe mode), and the system attempted to switch over to nominal mode.
• The firmware was enhanced to reduce the impact of heavy volume errors, which can be logged as "sender hang" errors.
• The firmware was enhanced to reduce the number of "retry fetch CE" and "DRAM spare" error logs entries that call out memory DIMMs.
• A problem was fixed that caused the first processor module in a node to be erroneously called out if an over-temperature condition was detected, instead of the processor module that was reporting the over-temperature condition.
• The firmware was enhanced to handle the I/O hub ISR (Integrated Switch Router) link port errors as software-recoverable, rather than as hard failures.  Before this enhancement, the links would have been guarded out even though these errors were recoverable.
• A problem was fixed that caused a service processor kernel panic due to an out-of-memory condition, with SRC B181720D.
  

• On systems with F/C 5708 and 5270 Dual port 10GB Ethernet adapter cards installed, a problem was fixed that caused SRC B7006970 to be erroneously logged when the card was powered on.
• In asymmetric and cross-coupled topologies, if there are no direct dlink connections between a storage drawer and a compute supernode (either through fail-in-place or through having a compute drawer or drawers at standby), then the storage drawer, upon restart or re-initialization of the lnmc daemon (lnmcd), does not provide a failover route to the target compute supernode even though there are suitable bounce points within the compute sub-cluster that can provide the indirect route.  The firmware was enhanced to provide this indirect route.
  

04/12/12

New Features and Functions

• Support for cross-coupled compute-to-storage topology for a 2 drawer storage sub-cluster.
  
• Support for cross-coupled compute-to-storage topology for a 4 drawer storage sub-cluster.
  

System firmware changes that affect all systems

• The firmware was enhanced to allow a node to continue to boot when unrecoverable SRC B181B70C is logged.
  
• A problem was fixed that caused an extraneous error log entry calling out DCCA-B and hub R5 when power was removed from DCCA-A, and the service processor and TPMD in DCCA-A were primary.
• The firmware was enhanced to more gracefully handle the system shutdown that is required when a hypervisor hang condition was encountered.  SRCs B7000602, B182951C, B1813918 and A7001151 were logged, and a service processor failover occurred, when the hypervisor hang condition and subsequent system crash occurred.
  
• The firmware was enhanced to cause the secondary service processor to automatically pick up configuration changes stored on the primary service processor.  This prevents the new configuration information from being lost if a service processor failover occurs before the secondary has picked up the new configuration information; typically this problem will only be encountered just after a system is installed.
• The firmware was enhanced to gracefully recover, and log the correct error logs, if the secondary DCCA loses power.
• A problem was fixed that prevented communication between the compute and storage networks in asymmetric ISR network topologies.  This affected network topologies DD2_64_8_2A, DD2_64_8_2B, DD2_64_8_4A, and DD2_64_8_4B.
  
• A problem was fixed that caused SRC B181E6F1 ("RMGR_PERSISTENT_EVENT_TIMEOUT") to be erroneously logged.
• The firmware was enhanced to reduce the number of memory DIMMs replaced due to correctable errors being logged.
• A problem was fixed that caused unrecoverable SRC B130CD03 to be erroneously logged.
• A problem was fixed that caused SRC B7000602 to be erroneously logged at power on.
• The firmware was enhance to prevent a potential deadlock in the opposite-side storage drawer if all of the cross-coupled dlinks between a compute supernode (at runtime) and a storage drawer (at runtime) are taken down.  This problem also affects indirect routing from compute to storage over cross-coupled links.
  
• A problem was fixed that caused the Local Network Management Controller (LNMC) to be set to the wrong state during a service processor (DCCA) fail-over.  If this problem occurs, the most likely symptom will be a communication failure on the ISR network.
• A problem was fixed that caused a partition running AIX to crash.
• A new level of optical link firmware is included in this service pack, and the optical link firmware update function is enabled.  The new optical link device firmware will be automatically installed the next time the node is booted after this service pack is installed.  Please see "Additional Details About Installing This Service Pack" in the "Important Information" section.
• The firmware was enhanced to increase the threshold of soft NVRAM errors on the service processor to 32 before SRC B15xF109 is logged.  (Replacement of the service processor is recommended if more than one B15xF109 is logged per week.)