IBM Platform Symphony 7.1.0 Interim Fix 413025 Readme File

Abstract

MapReduce service logs should automatically rotate.

Description

The MapReduce service logs use a lot of space on each data node, and must be deleted manually daily. This fix allows you to configure the MapReduce service logs to auto-rotate.

 

To enable this fix, the environment variable “ENABLE_MAPREDUCE_LOG_ROTATION” need to be set as “Y” or “y” in the MapReduce application profile.

This fix applies to IBM® InfoSphere BigInsights 4.1 Linux 64-bit.

Readme file for: IBM Platform Symphony

Product/Component Release: 7.1 Fix Pack 1

Update Name: Interim Fix 413025

Fix ID: sym-7.1-build413025-iop

Publication date: 20 July 2016

Last modified date: 20 July 2016

Contents:

1.     List of fixes

2.     Download location

3.     Products or components affected

4.     Installation and configuration

5.     List of files

6.  Copyright and trademark information

1.   List of fixes

APAR P101790

2.   Download location

Download Fix 413025 from the following location: http://www.ibm.com/eserver/support/fixes/

3.   Products or components affected

Product/Component Name:

Platform Symphony/MapReduce SIM

4.   Installation and configuration

4.1          Before installation

1.  Log on to the symphony master host as the cluster administrator and disable all applications:

> source $EGO_TOP/cshrc.platform
> egosh user logon -u Admin -x Admin
> soamcontrol app disable all

2.  Stop all services except Symphony from the Ambari GUI.

3.  Back up files.

Log on to every management and compute host in the cluster and back up the following files that will be replaced by this fix to other directory:

> $PMR_HOME/integration/IBM_BigInsights_4.1/hadoop-mapreduce-client-core-plus-HDP2.3_2.7.1.jar

> $PMR_HOME/7.1/linux2.6-glibc2.3-x86_64/lib/hadoop-2.7.x/hadoop-mapreduce-client-core-2.7.1.jar

> $PMR_HOME/../7.1/linux2.6-glibc2.3-x86_64/etc/sim

4.2          Installation steps

1.  Log on to every management and compute host in the cluster and decompress the sym7.1-build413025.tar.gz package:

> tar zxfo sym7.1_lnx26-lib23-x64_build413025.tar.gz -C $EGO_TOP

             2.  Modify RunMapReduceService.sh

Modify the code from:

#!/bin/bash

filepath=${PMR_HOME}/logs/${USER}.pmr.service.${log4cxx_appname}.${log4cxx_autoindex}.log

to:

#!/bin/bash

if [ $ENABLE_MAPREDUCE_LOG_ROTATION = "Y" ] || [ $ENABLE_MAPREDUCE_LOG_ROTATION = "y" ]; then

filepath=${PMR_HOME}/logs/${USER}/pmr.service.${log4cxx_appname}.${log4cxx_autoindex}.log

else

filepath=${PMR_HOME}/logs/${USER}.pmr.service.${log4cxx_appname}.${log4cxx_autoindex}.log

fi

3.  Enable this fix

Configure the ENABLE_MAPREDUCE_LOG_ROTATION environment variable as “Y” or “y” in the MapReduce application profile, as follows:

                 <SIM startUpTimeout="120">

                     <osTypes>

                         <osType name="X86_64" startCmd="${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/etc/sim">

                             <env name="SOAM_HOME">${EGO_TOP}/soam</env>

                             <env name="ENABLE_MAPREDUCE_LOG_ROTATION">Y</env>

                         </osType>

                     </osTypes>

                 </SIM>

4.  Register and enable the application

> soamreg MapReduce7.1.xml

> soamcontrol app enable MapReduce7.1

4.3          After installation

1. verify the installation

Run:

> sim –V

The following is an example output of successfully running the command:

IBM Platform Symphony 7.1.0.0 build 413025, Jul 06 2016

Copyright International Business Machines Corp, 2001-2014.

US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.    

2. After deploying the package, perform the Symphony integration from the Ambari GUI.

3. Restart Symphony from the Ambari GUI.

4. Start all services from the Ambari GUI.

5.   Log on to each host as root and add two parameters to the log4j.properties file, located in /etc/hadoop/conf folder:

log4j.appender.pmr.service.MaxFileSize=10000KB

log4j.appender.pmr.service.MaxBackupIndex=10

where:

MaxFileSize: The output file is allowed to be rolled over to backup files when file size is larger than this maximum size value.
The MaxFileSize option takes a long integer in the range 0 - 2^63. You can specify the value with the suffixes "KB", "MB" or "GB" so that the integer is interpreted being expressed respectively in kilobytes, megabytes or gigabytes. For example, the value "10KB" will be interpreted as 10240.

MaxBackupIndex: the maximum number of backup files to keep around, the MaxBackupIndex option determines how many backup files are kept before the oldest is erased. This option takes a positive integer value. If set to zero, then there will be no backup files and the log file will be truncated when it reaches MaxFileSize.

There is one backup file by default.

Note:

The above  two configuration parameters must be set after starting the cluster; otherwise, they will be removed if they are set before starting IBM Open Platform (IOP).

6.  Log on to the master host as the cluster administrator and enable application:

> source $EGO_TOP/cshrc.platform
> egosh user logon -u Admin -x Admin
> soamcontrol app enable <AppName>

4.4          Uninstallation

1.  Log on to the symphony master host as the cluster administrator and disable all applications:

> source $EGO_TOP/cshrc.platform
> egosh user logon -u Admin -x Admin
> soamcontrol app disable all

2.  Stop all services except Symphony from the Ambari GUI.

3.   Restore the backup files.

Log on to every management and compute host in the cluster and restore the backup files:

> $PMR_HOME/integration/IBM_BigInsights_4.1/hadoop-mapreduce-client-core-plus-HDP2.3_2.7.1.jar

> $PMR_HOME/7.1/linux2.6-glibc2.3-x86_64/lib/hadoop-2.7.x/hadoop-mapreduce-client-core-2.7.1.jar

> $PMR_HOME/../7.1/linux2.6-glibc2.3-x86_64/etc/sim

             4.  Modify RunMapReduceService.sh

#!/bin/bash

filepath=${PMR_HOME}/logs/${USER}.pmr.service.${log4cxx_appname}.${log4cxx_autoindex}.log

5.  Disable this fix

Clear ENABLE_MAPREDUCE_LOG_ROTATION environment variable in the MapReduce application profile.

6.  Reregister and enable the application

> soamreg MapReduce7.1.xml

7. Perform the Symphony integration from the Ambari GUI.

8. Restart Symphony from the Ambari GUI.

9. Start all services from the Ambari GUI.

10.  Log on to the master host as the cluster administrator and enable application:

> source $EGO_TOP/cshrc.platform
> egosh user logon -u Admin -x Admin
> soamcontrol app enable <AppName>

5.   List of files

hadoop-mapreduce-client-core-plus-HDP2.3_2.7.1.jar

hadoop-mapreduce-client-core-2.7.1.jar

sim

6.   Copyright and trademark information

© Copyright IBM Corporation 2016

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM®, the IBM logo and ibm.com® are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.