SAS Adapter/Controller (CCIN 57B5, 57BB, 57C3, 57C4) Microcode README   IBM Power Systems PCIe2 1.8GB Cache RAID SAS Adapter Tri-port 6Gb (CCIN 57B5, 57BB) PCIe2 3.1GB Cache RAID SAS Enclosure 6Gb x8 (CCIN 57C3) PCIe2 RAID SAS Adapter Dual-port 6Gb (57C4)    DATE:  July 18, 2016   Table of Contents 1.0 Microcode Change History 2.0 General Information 2.1 Installation Time 2.2 Machine's Affected 2.3 AIX Requirements 2.4 Linux Requirements 2.5 IBM i Requirements 3.0 Determine the Current Microcode Level for AIX 3.1 Determine the Current Microcode Level for Linux 4.0 Downloading and Unpacking the Firmware Update Package from CORE 4.1 Downloading the RPM Format File to the Target Server 4.2 Microcode Update via CD 5.0 Verifying microcode before download and notes 5.1 Microcode Download Procedure-SAS Adapters for AIX 5.2 Microcode Download Procedure-SAS Adapters for Linux 6.0 Microcode Readme Change History ===================================================================================== 1.0 Microcode Change History:   0350005d Fixes for: -Possible adapter machine check may result in SRN57B58151 SRN57BB8151 SRN57C38151 SRN57C48151 -Incorrect data check recovery may result in an adapter Unit Check SRN57B58151 SRN57BB8151 SRN57C38151 SRN57C48151 -Possible hard command timeout resulting in SRN57B53109 SRN57BB3109 SRN57C33109 SRN57C43109 and/or SAS_ERR4 0350005a -HIPER: Fix for - Incorrect error recovery of some SAS and certain internal hardware errors in a Multi-initiator and high availability (Dual Controller) configuration which could result in a system hang and lead to other unpredictable results, including but not limited to, file system failures and failure of the system to IPL requiring data to be restored for recovery. SRNxxxxFFFD, SRNxxxx4100, SRNxxxx8130, SRNxxxx4060, SRNxxxx4040, SRNxxxxFFFE, SRNxxxx8151, SRNxxxx8150 03500058 -Error Logging correction to properly log internal exceptions. SRC57B58151 SRC57BB8151 SRC57C38151 SRC57C48151 -Enhancement to block unsupported 4k sector devices to prevent SRC57B58300 SRC57BB8300 SRC57C38300 SRC57C48300 03500052 -Adapter error recovery for SAS fabric hardware failures may result in SRC57B58151 SRC57BB8151 SRC57C38151 SRC57C48151 -Enhanced error recovery for dual storage IOA configurations -Enhanced device error recovery -Some error recovery during dual Storage IOA communication may cause an extended timeout resulting in SCRB6005123 -Device error reporting invalid max capacity on certain SSD's may result in SRC57C38151 0350004c -Incorrect handling of a device or SAS failure may result in SRC57B58151 SRC57BB8151 SRC57C38151 SRC57C48151 -An array failure during a cache flush may result in an SRC57B58151 SRC57BB8151 SRC57C38151 -Parity updates may get out of sync between dual storage IOA's resulting in possible SRC57B59041 SRC57C39041 SRC57C49041 SRC57BB9041 -False error condition detected when operating in a cache full condition and encountering a SAS recoverable event, may result in SRC57B58151 SRC57BB8151 SRC57C38151 SRC57C48151 -Multiple error logging enhancements -Multiple adapter serviceability enhancements -Adapter error recovery enhancements in dual storage IOA configurations -Multiple error recovery enhancements 03500047 -Enhancements in adapter error recovery to prevent possible failure modes resulting in the adapter not able to recover and leaving the dual storage IOA pair running in a performance degraded mode. SRC57B58150, SRC57C48150, SRC57C38150, SRC57FF8156 03500041 -GA level (Base support) for Feature ESA3, CCIN 57BB -Base adapter support for using IBM i 520 byte sector disk through VIOS Additional requirements (for the IBM i 520 byte support through VIOS):  VIOS (2.2.3.0) & IBMi Licensed Internal Code V7R1m0 (TR 7) Note: the aforementioned IBM i & VIOS levels are NOT required to install this adapter microcode -During error recovery an invalid index may result in 04448400 or SRC57B58151, SRC57C38151, SRC57C48151. 0350003c -04448400 (SRC57B58151, SRC57C48151, SRC57C38151):     --Incorrect SAS error recovery in a Dual Storage IOA configuration     --In a mirrored configuration with caching adapters, incorrect handling of an internal adapter operation     --In a RAID-6 configuration, incorrect handling of an internal adapter operation -SRCA6000255: FC5913, FCESA1, FCESA2, FCEDR1 adapter may become unresponsive during extended periods of inactivity (i.e. applying group PTFs or other PTF packages) -06678300 (SRC57B54040, SRC57C44040, SRC57C34040): May get logged during expander/SES firmware microcode update -0110A000 or 0110A100 (SRC57B5FFFD, SRC57C4FFFD, SRC57C3FFFD): Incorrect enforcement of rebuild progress boundary during device parity rebuild -06678100 (SRC57B59074, SRC57C49074, SRC57C39074): Following a successful device parity rebuild the Asymmetric error may incorrectly be logged -01448200 or 01109000 (SRC57B5FF3D, SRC57C4FF3D, SRC57C3FF3D): Incorrect hardware checker may result in false error -Error logging enhancements -Starting device parity with hotspare may fail in rare configurations -A replacement of a failed mirrored device in a dual storage adapter configuration may result in a false reclaim cache being required. 015000ab -04448400 resulting from -Drive failure in a Dual Storage IOA configuration -Including and excluding devices in existing RAID arrays -Incorrect SAS error recovery in a Dual Storage IOA configuration -04440000 resulting from incorrect SAS error recovery which would require a sector reallocate on certain drive types -Error logging enhancements -Performance improvements for query commands issued to IOA 015000a8 -SAS I/O adapter enhancements during error recovery to prevent a very slight chance of undetected data corruption. 015000a1 Fixes for -Incorrect handling of R/W protect and STOP (IBM i only) state -Cache sync hang -Hang due to out of page resources -Enhanced SAS error recovery to prevent host timeouts -IOASC 04448400     --Incorrect error recovery during a RAID-5 cache destage     --An abnormal reset during a device parity rebuild -IOASC 02048000 and/or 066B8100     --Incorrect timeout value on adapter to adapter commands in a Dual Storage IOA Configuration 0150009a -GA level for CCIN 57C3, 57C4 -HIPER: Fix for UECC error which can cause the F/W cache to become disabled. -Significant performance enhancements for RAID-5 and mirroring configurations -General device error recovery enhancements in a Dual Storage IOA Configuration to prevent certain cases of IOASC 0108000, 0408000, 04448400 -Fixes for certain Watchdog Timeout errors -PRC 04440000, DMGR logging failure for Abort Task Set TMF Enhanced error recovery -Properly log device failure error prior to reset for switchover on Secondary. Error reporting enhancements to log necessary errors for certain device failures in a Dual Storage IOA Configuration -Queue Full mitigation -Fixes for Adapter Reset/Reload IOASC 04448200, 04448400    - PRC 80FF0300, TMF processing could get out of sync if a device goes not connected when a TMF is outstanding)    - Lost ops due to bridge handling cause soft cmd timeouts    - PRC 80FF0300, double unit check processing command timeout    - Incorrect WWID being obtained at IPL for remote IOA    - PRC 80FF0300, when logging T10 errors with single index mode instead of a page table list    - Certain cases of a drive failure within a RAID array in a Dual Storage Controller configuration    - Array Manager is taking too long to update footprints when doing switchover for large arrays. Possible host timeout due to extended error recovery time with large RAID arrays    - PRC 10013082, IE Leak. Incorrect memory resource management in the adapter may result in lack of available control store elements 01500090 - Level 0150008d + 1 Additional fix for 04448200/15212205 Battery Errors. 0150008d Fixes: -SAS I/O adapter enhancements during error recovery to prevent a very slight chance of undetected data corruption. -Adapter Unit Check (04448400) for     *Incorrect abort request handling     *Loss of access to System Enclosure Services -Incorrect error logging when corrupted parity is detected in a RAID-6 array may prevent 06698200 from being surfaced -General error recovery enhancements to prevent certain Temporary Error Conditions (FFFE). These may have surfaced as VRSAS_ERR21T -Stopping multiple RAID-5 or RAID-6 arrays at the same time may result in 04448400 (IBMi only) Performance Enhancements: -Enhancements to performance information reported by the adapter. -Possible performance impact with certain sequential workloads to Solid State Devices and may result in extended IPL time. Prerequisite - Prerequisite for FC 5913 and FC 5887 in Mode 2 configurations 01500087 -01500087 contains all the fixes in 01500084 plus an additional fix for the below issue: Fix to prevent Adapter Unit Check if device reports back a maxLBA > 32 bits or an unexpectedmaxLBA of 0 in Read Capacity. The fix will now correctly fail the device. 01500084 -01500084 contains all the fixes in 01500080 plus an additional fix for the below issue: Fixed undetected Data Corruption condition found in IBM i environment       -No exposure in Dual Controller (cached) mode       -Exposure only with 01500080 code       -Exposure only if running in non-cached mode. 01500080 Fixes for -04448400 conditions     -Unique drive failure mode where the drive recovers after failing     -Stopping multiple RAID arrays (IBM I only) simultaneously -SRC B6005120     -Insufficient adapter resources may lead to command timeouts resulting in reset/reload of the adapter -Potential system hang due to incorrect timing on Dual Storage IOA link -Possible command timeout under intensive write workload Error logging enhancements 0150007a -GA Level for CCIN 57B5. -01500077 + 2 fixes 1) Possible cache data store clearing at IPL due to am improper setting in the "allowed resets" bit in DRAM Controller (very low probability 2) Enhanced error data when logging battery errors 01500077 May exist on some CCIN 57B5 adapters.  Any adapters at this level should be updated to 0150007a or beyond ASAP   2.0 General information This subject microcode is to be used on the following SAS Adapter/controller types:   CCIN 57B5, 57BB: PCIe2 1.8GB Cache RAID SAS Adapter Tri-port 6Gb CCIN 57C3: PCIe2 3.1GB Cache RAID SAS Enclosure 6Gb x8 CCIN 57C4: PCIe2 RAID SAS Adapter Dual-port 6Gb   2.1 Installation Time: Approximately 30 minutes   2.2 Machine's Affected: See Universal Sales Manual or other relevant documents for current listings of supported systems for these adapters   2.3 AIX Requirements: TL refers to Technology Level SP refers to Service Pack   57B5 6.1 TL3 SP10 6.1 TL4 SP10 6.1 TL5 SP6 6.1 TL6 SP5 6.1 TL7 7.1 TL0 SP3 7.1 TL1 or above   57BB 6.1 TL7 SP9 6.1 TL8 SP4 6.1 TL9 7.1 TL1 SP9 7.1 TL2 SP4 7.1 TL3 VIOS 2.2.3.0 or above   57C3 & 57C4 6.1 TL5 SP8 6.1 TL6 SP7 6.1 TL7 SP4 7.1 TL0 SP5 7.1 TL1 SP4 or above   2.4 Linux Requirements: 57B5 & 57C4 Red Hat Enterprise Linux 5.7 for POWER, or later Red Hat Enterprise Linux 6.1 for POWER, or later Novel SUSE Linux Enterprise 10, Service Pack 4, or later Novel SUSE Linux Enterprise 11 Service Pack 1, or later   57BB Red Hat Enterprise Linux 5.7 for POWER, or later Red Hat Enterprise Linux 6.1 for POWER, or later Novel SUSE Linux Enterprise 10, Service Pack 4, or later Novel SUSE Linux Enterprise 11 Service Pack 2 or later   57C3 Red Hat Enterprise Linux 5.8 for POWER, or later Red Hat Enterprise Linux 6.3 for POWER, or later Novel SUSE Linux Enterprise 11 Service Pack 2, or later   2.5 IBM i Requirements: 57B5 IBM i 7.1 with i 7.1.0 machine code: RS710-10 OS & (RS710-D LIC or Technology Refresh 3 (TR PTF Group SF99707 Level 3)) IBM i 6.1 with i 6.1.1 machine code - i 6.1.1: RS610-10 OS & RS611-E LIC   57C3 Not supported on IBM i   57BB, 57C4 IBM i 7.1 (& applicable PTFs), or later IBM i 6.1 with 6.1.1 machine code 6.1.1  (& applicable PTFs) or later     3.0 Determine the Current Microcode Level for AIX: Before you install the microcode, it is important to determine the microcode level of the Adapter installed in the target system. Use the following instructions to read the ROM level stored in the Adapter's VPD. 1)  List the SAS RAID Adapters installed in the system by typing:     lsdev -C | grep sissas Note: All of the SAS RAID initiators addresses that are installed will be displayed. The names will besissasX, where X is 0, 1, etc. 2) To check the current microcode level for the adapter or controller enter the following command    lscfg -vl sissasX Where X is the number/s of the previous command. The Command will produce output similar to   Example show for CCIN 57B5 DEVICE   LOCATION                                     DESCRIPTION sissas1 U78C0.001.DBJ0102-P2-C4-T1 PCIe2 1.8GB Cache RAID SAS Adapter Tri-port 6Gb PCIe2 1.8GB Cache RAID SAS Adapter Tri-port 6Gb : Part Number.................74Y7759 FRU Number..................74Y7759 Serial Number...............YL30JH116130 Manufacture ID..............00JH EC Level....................0 ROM Level.(alterable).......01500077 Customer Card ID Number.....57B5 Product Specific.(Z1).......1 Product Specific.(Z2).......2D20 Hardware Location Code......U78C0.001.DBJ0102-P2-C4-T1   If the ROM Level is less than 0350005d then you should update the microcode.   3.1 Determine the Current Microcode Level for Linux: 1) Type "iprconfig" 2) Select "Display hardware status" 3) Select SAS adapter by typing a "1" in front of it and pressing Enter. 4) The Command will produce output similar to "Firmware Version . . . . . . . . . . . . : 01500077" If the Firmware Version is less than 0350005d then you should update the microcode. 4.0 Downloading and Unpacking the Firmware Update Package from CORE NOTE:  The CORE image will now be the RPM Format File. 1) After downloading the files from CORE on to your personal system find the following files in the CORE destination directory (c:\ibmdocs\DISKIMG):     A) "pci.1014033d.50-0350005d-1.aix6.1.noarch.rpm"     B) "pci.1014033D.50-0350005d-1.Linux.noarch.rpm" 2) Log in to the Target Server 3) Create a directory, if not already existent (mkdir /tmp/microcode; mkdir / tmp/microcode/RPM) 4) Transfer the file to the Target Server "/tmp/microcode/RPM directory" using ftp or another method.  Change to that directory, "cd /tmp/microcode/RPM". 5a) For AIX/Linux:  Unpack the RPM file by using the following command:  "rpm -ihv --ignoreos --force pci.1014033d.50-0350005d-1.aix6.1.noarch.rpm" 5b) For Linux:  Unpack the RPM file by using the following command:  "rpm -ihv --ignoreos --force pci.1014033D.50-0350005d-1.Linux.noarch.rpm" 6a) For the AIX rpm file:  The microcode files will be added to /usr /lib/microcode/. 6b) For the Linux rpm file:  The microcode files will be added to /lib/firmware/. 4.1 Downloading the RPM Format File to the Target Server Use this method to download to an AIX or Linux system: Note: The instructions that follow are specific AIX/Linux commands. AIX/Linux commands are CASE (lower and upper) SENSITIVE, andmust be entered exactly as shown, including filenames. 1) Make a directory on an AIX/Linux system to receive the RPM format file.     Enter: "mkdir /tmp/microcode"     Enter: "mkdir /tmp/microcode/RPM" 2) Transfer the RPM format file to the /tmp/microcode/RPM directory (using "Save as ...").  Change to that directory, "cd /tmp/microcode/RPM". You'll see the filenames for the RPM files.     A) For AIX/Linux:  The filename is pci.1014033d.50-0350005d-1.aix6.1.noarch.rpm     B) For Linux:  The filename is pci.1014033D.50-0350005d-1.Linux.noarch.rpm 3) Unpack the file by executing the instructions below:     A) For AIX/Linux:  Type "rpm -ihv --ignoreos --force pci.1014033d.50-0350005d-1.aix6.1.noarch.rpm"     B) For Linux:  Type "rpm -ihv --ignoreos --force pci.1014033D.50-0350005d-1.Linux.noarch.rpm" 4a) For AIX rpm file:  The microcode files will be added to /usr /lib/microcode/. 4b) For Linux rpm file:  The microcode files will be added to /lib/firmware/.   4.2 Microcode Update via CD Below are references and the general flow on how to update microcode update via CD 1) Boot the system via a Standalone Diagnostic's CD b) To create a bootable Diagnostic's CD go to the following URL. Burn cdlatest.iso on to a CD-R (or CD-RW). Currently, DVD media cannot be utilized. http://www14.software.ibm.com/webapp/set2/sas/f/diags/home.html c) After booting to Standalone Diagnostics, the Diagnostic's CD can be removed. 2) The Diagnostic's CD should be replaced by a CD containing the unpacked microcode image(s). The CD containing the microcode images should be prepared prior. This can be done by taking the rpm file(s) from the microcode download site and unpacking them on a usable server. The unpacked files can then be burned on to the CD media. b) Another method is to use a Microcode Update CD See the below website for more details http://www14.software.ibm.com/webapp/set2/firmware/lgjsn?mode=10&page=cdrom.html#downloadcdrom The Microcode Update CD and Discovery Tool can also be ordered through the following link http://www14.software.ibm.com/webapp/set2/firmware/lgjsn?mode=10&page=compare.html&title=Microcode%20Survey%20and%20update%20tool 5.0 Verifying microcode before download and notes NOTE: Microcode download must be performed separately for each Adapter bus under distinct Logical Partitions (LPAR's).               Please save current and older versions of the microcode update files in case you need to restore your system. NOTE: Microcode Download CAN be performed concurrently. Installation Time:  Approximately 30 min System Time. Please verify the file size and checksum of the raw microcode files matches the below. Filesize:  2624264 Checksum:  13164 For AIX/Linux:  "ls -l /usr/lib/microcode/53495350.0350005d" to verify Filesize.                  "sum /usr/lib/microcode/53495350.0350005d" to verify Checksum. For Linux:  "ls -l /lib/firmware/pci.1014033D.50.0350005d" to verify Filesize.                  "sum /lib/firmware/pci.1014033D.50.0350005d" to verify Checksum. For systems that have LPAR installed, the microcode file must be placed on all partitions that have a SAS device in order to update the code.  Follow this procedure for each partition: For AIX/Linux:     A)  Log into the target partition.     B)  Type: "cd /usr/lib/microcode"     C)  FTP (for AIX) or SFTP (for Linux) to the partition that has the media bay assigned. (ie.  ftp/sftp partition1.ibm.com)     D)  Type: "cd /usr/lib/microcode"     E)  If using FTP then Type: "bin"     F)  Type: "get 53495350.0350005d"     G)  Type: "quit" For Linux:     A)  Log into the target partition.     B)  Type: "cd /lib/firmware"     C)  SFTP to the partition that has the media bay assigned. (ie.  "sftp partition1.ibm.com")     D)  Type: "cd /lib/firmware"     E)  Type: "get pci.1014033D.50.0350005d"     F)  Type: "quit" 5.1 Microcode Download Procedure-SAS Adapters for AIX Procedure: 1) It is advised, but not necessary to stop all applications. 2) Enter diag 3) Select the "Task Selection" from diagnostics menu. 4) Select "Download Microcode" or "Microcode Tasks -> Download Microcode" from "Task Selection" menu. 5) Select all of the SAS Adapters/Controllers that need to be updated from the list of devices by pressing "Enter".  The devices will be displayed assissas #, where # is the number of the device.  Press "F7". 6) A dialogue box will be displayed on screen.  Press "Enter" to continue. 7) Select "/usr/lib/microcode". 8) A dialogue box will may be displayed on screen.  It will state that the current microcode level on the adapter is not in the /usr/lib/microcode directory.  This is acceptable as you will be flashing new microcode onto the adapter.  Also, incorrect code will automatically be rejected.  Press "Enter" to continue. 9) An "M" will be displayed next to the latest level of microcode in the /usr /lib/microcode directory.  Select that level and press "Enter". 10) The following message will appear on the screen when download is completed: "Microcode download complete successfully.  The current microcode level for the devicesissas# is .....  Please run diagnostics on the adapter to ensure that it is functioning properly."  DO NOT run diagnostics on the device at this time. 11) Exit diagnostics. 12) Verify the code level is 0350005d by typing "lscfg -vl sissas#" for each SAS device updated. 5.2 Microcode Download Procedure-SAS Adapters for Linux Procedure: 1) Stop all applications. 2) Type "iprconfig" and press "Enter". 3) Select "Download Microcode" and press "Enter". 4) Select all of the SAS Adapters/Controllers  that need to be updated from the list of devices by pressing "1". Then press "Enter". 5) Select the appropriate microcode image to flash by pressing "1".  (The image will end in 0350005d).  Press Enter.  Press Enter again to confirm. 6) If multiple devices were selected in step 4, then repeat step 5 until finished. 6.0 Microcode Readme Change History 2016-07-18 -Added microcode level 0350005d 2015-02-25 -Added microcode level 0350005a 2014-10-07 -Added microcode level 03500058 2014-07-31 -Added microcode level 03500052 2014-01-31 -Added microcode level 03500047 2013-10-14 -Added microcode level 03500041, CCIN 57BB. 2013-09-19 -Added microcode level 0350003c 2013-01-29 -Added microcode level 015000ab 2012-10-11 -Added microcode level 015000a8 2012-07-24 -Added microcode level 015000a1 2012-06-27 -Added UECC error to 0150009a microcode details 2012-06-04 -Added microcode level 0150009a 2012-04-04 -Added microcode level 01500090 2012-03-14 -Added microcode level 0150008d 2012-02-12 -Added microcode level 01500087 2011-12-08 -Added microcode level 01500084 2011-11-16 -Added microcode level 01500080 2011-10-21 -First Readme File for microcode level 0150007a