Readme File for IBM^® Spectrum Symphony 7.1.2 and IBM^® Spectrum Conductor with Spark 2.2.1 Interim Fix 484101

Readme File for: IBM Spectrum Symphony and IBM Spectrum Conductor with Spark

Product Release: 7.1.2 and 2.2.1

Update Name: Interim Fix 484101

Fix ID: sym-7.1.2-cws-2.2.1-build484101-jpmc

Publication Date: Mar 29, 2018

This interim fix provides the following fixes for a cluster with IBM Spectrum Symphony 7.1.2 and IBM Spectrum Conductor with Spark 2.2.1 installed:

·             Arbitrary Remote Code Execution (RCE)

·             Inadequate permissions for notebook data directories

·             Vertical authorization bypassing on the Symping page

·             XML entity expansion (XXE)

·             Cross-site Scripting Stored (XSS)

·             Arbitrary file reading

·             Clear the text of the user passwords reflected in the cluster management console

·             User enumeration

·             Host lost in 'egosh rg' resource list after vemkd restart

·             VEMKD is slow when processing DISTRIBUTE requests

·             SD and SSM calls chmod on the GPFS root directory

 

This interim fix also provides RFE 115693: Request for Spark 2.1.1, to merge the feature of disabling the terminal menu for the Jupyter 4.1.0 notebook to IBM Spectrum Conductor with Spark 2.2.1.

Contents

1. List of fixes

2. Download location

3. Product and components affected

4. Installation and configuration

5. Uninstallation

6. List of files

7. Copyright and trademark information

1.    List of fixes

APAR: P102505, P102496, P102460, P102468

2.    Download location

Download Interim Fix 484101 from the following location: https://www.ibm.com/eserver/support/fixes/

3.    Product and components affected

Component name, Platform, Fix ID:                                                    

PMC/REST/EGO/SOAM/Jupyter-4.1.0, Linux x86_64, sym-7.1.2-cws-2.2.1-build484101-jpmc

4.    Installation and configuration

Follow the instructions in this section to download and install this interim fix in your cluster.

System requirements

Linux x86_64

Before installation

1.     Log on to the master host as the cluster administrator, disable applications, and stop the following services:

egosh user logon -u Admin -x Admin

soamcontrol app disable all

egosh service stop WEBGUI REST SD RS

2.     For recovery purposes, back up the following files:

cd $EGO_TOP

tar -cvf backup.tar gui/conf/useracl/permission_GUIPermissionSoam.acl

tar -uf backup.tar gui/3.6/lib/rest-ego-3.6.0.jar

tar -uf backup.tar gui/3.6/lib/soamgui.jar

tar -uf backup.tar gui/3.6/lib/egogui.jar

tar -uf backup.tar gui/3.6/lib/commons-ego.jar

tar -uf backup.tar gui/3.6/lib/newegogui.jar

tar -uf backup.tar wlp/usr/servers/gui/apps/perf/3.6/perfgui/WEB-INF/classes/com/platform/perf/report/action/ReportAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/wizard/CreateAppWizardOperator.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/application/RegisterAppXmlAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationRestResource.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationV1RestResource.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/action/applications/AddApplicationAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/action/appConf/ImportAppProAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/action/appConf/ExportAppProfileAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/lib/soamgui.jar

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/api/symping/SympingConfSetting.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/wizard/CreateAppWizardOperator.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/wizard/importAppProActionWizard.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/srvpackage/ExportDeploymentXmlAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/srvpackage/ConfigurePackageProfileAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/appConf/GetApplicationConfAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/appConf/ImportAppProAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/appConf/ExportAppProfileAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/common/FormatXMLAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/application/RegisterAppXmlAction.class

tar -uf backup.tar wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/symping/symping.jsp

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/wizard/CreateAppWizardOperator.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/application/RegisterAppXmlAction.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationRestResource.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationV1RestResource.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/lib/soamgui.jar

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/wizard/CreateAppWizardOperator.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/application/RegisterAppXmlAction.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationRestResource.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationV1RestResource.class

tar -uf backup.tar wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/lib/soamgui.jar

tar -uf backup.tar wlp/usr/shared/resources/rest/3.6/commons-ego.jar

tar -uf backup.tar 3.6/linux-x86_64/bin/egosh

tar -uf backup.tar 3.6/linux-x86_64/etc/rs

tar -uf backup.tar 3.6/linux-x86_64/etc/vemkd

tar -uf backup.tar 3.6/linux-x86_64/lib/jni/libVEMApiCommon.so

tar -uf backup.tar soam/7.1.2/linux-x86_64/bin/soamdeploy

tar -uf backup.tar soam/7.1.2/linux-x86_64/etc/sim

tar -uf backup.tar soam/7.1.2/linux-x86_64/etc/ssm

tar -uf backup.tar soam/7.1.2/linux-x86_64/etc/sd

3.     Back up the $EGO_CONFDIR/../../gui/conf/navigation/pmc_menu.xml file to another directory.

Installation

1.     Log on to any management and compute host in your cluster as the cluster administrator and decompress the sym-7.1.2.0-cws-2.2.1.0_x86_64-build484101.tar.gz file to the directory where you installed IBM Spectrum Symphony and IBM Spectrum Conductor with Spark.

For example, run:

tar zxfo sym-7.1.2.0-cws-2.2.1.0_x86_64-build484101.tar.gz -C $EGO_TOP/

2.     Delete the following files if they exist:

$EGO_TOP/wlp/usr/servers/gui/apps/ego/3.4/platform/generaltable/getDeviceInfo.jsp

$EGO_TOP/wlp/usr/servers/gui/apps/ego/3.5/platform/generaltable/getDeviceInfo.jsp

$EGO_TOP/wlp/usr/servers/gui/apps/ego/3.6/platform/generaltable/getDeviceInfo.jsp 

$EGO_TOP/wlp/usr/servers/gui/apps/soam/7.1.2/symgui/generaltable/getDeviceInfo.jsp

3.     Copy the following configuration file to your shared directory:

cp $EGO_TOP/gui/conf/useracl/permission_GUIPermissionSoam.acl $EGO_CONFDIR/../../gui/conf/useracl/

4.     Open the $EGO_CONFDIR/../../gui/conf/pmcconf/pmc_conf_ego.xml file and check the following sections to restrict log retrieval to a whitelist of directories for the host. Add them if they are not configured:

<Parameter>
   <Name>RestrictHostLogRetrieve</Name>
      <!-- This parameter restricts retrieving logs for a host. -->
      <!-- Valid values are true or false. By default, this parameter is false, so that there is no restriction: all logs from any directory can be retrieved. -->
      <Value>true</Value>
</Parameter>

<Parameter>
    <Name>WhitelistLogsDir</Name>
       <!--This parameter takes effect only when the RestrictHostLogRetrieve parameter is set to true. -->
       <!--This parameter allows you to define the specific directories from which logs can be retrieved for a host. -->
       <!--Define any number of directories as required, separating multiple directories by a semicolon (";"). -->
       <Value>${SOAM_HOME};${EGO_TOP}</Value>
</Parameter>

5.     Edit the $EGO_CONFDIR/../../gui/conf/navigation/pmc_menu.xml file to set the URL value as empty for standardReport and customReport items. For example:

<MenuItems id="eventAndReportTreeSource">

     <MenuItem id="standardReport" label="@{pmc.tree.node.standarReport.label}"

               status="" url="" layoutSourceId="standarReportLayout" tabGroup=""

               highlightTabId="" aclResource="Main_perf_standard" aclPermission="1"

               helpGroupId="standardreport" />

 

     <MenuItem id="customReport" label="@{pmc.tree.node.customerReport.label}"

               status="" url="" layoutSourceId="customReportLayout" tabGroup=""

               highlightTabId="" aclResource="Main_perf_custom" aclPermission="1"

               helpGroupId="customerreport" />

6.     Clean up the gui/work directories by deleting all subdirectories and files from the following directories:

rm -rf $EGO_TOP/gui/work/*

rm -rf $EGO_TOP/gui/workarea/*

rm -rf $EGO_TOP/kernel/rest/workarea/*

rm -rf $WLP_OUTPUT_DIR/webgui_hostname/gui/workarea/*

7.     Clear your browser cache.

8.     Start the WEBGUI, REST, SD and RS services:

egosh service start WEBGUI REST SD RS

9.     Update permissions for the Jupyter notebook.

Note: This update can be applied to one Spark instance group at a time; Spark instance groups that are not updated continue to work as is. After updating the notebook permissions, the permissions cannot be reverted. Spark instance groups apply only to IBM Spectrum Conductor with Spark.

a.     Stop the Spark instance group.

b.    Update your Jupyter notebook package:

                              i.        Decompress the $EGO_TOP/Jupyter-4.1.0.tar.gz package to a temporary directory, replace the Anaconda2-4.1.1-Linux-x86_64.sh package with your customized Anaconda package, and then regenerate the Jupyter-4.1.0.tar.gz package.

                             ii.        Copy the regenerated Jupyter-4.1.0.tar.gz package to a host with web browser access.

                            iii.        From the cluster management console, navigate to Workload > Spark > Notebook Management.

                            iv.        Select Jupyter and click Configure.

                             v.        Click Browse and locate the Jupyter notebook package that you copied in step 9b-ii,

                            vi.        Modify the Start command field to add “ --disable_terminal true”. For example:

./scripts/start_jupyter.sh --disable_terminal true

                           vii.        Remove the Prestart Command and leave it empty.

                          viii.        Click Update Notebook.

                            ix.        Select Workload > Spark > Spark Instance Group.

                             x.        Apply the package update for the Spark instance group by clicking the corresponding Available: number link, and then click Update.

c.     Restart the Spark instance group after the package has finished updating.

10.  Edit the $EGO_CONFDIR/ego.conf file to enable the EGO_ENABLE_UNAVAILABLE_HOST_IN_RG parameter as “Y” or “y”.

11.  From the master host, restart EGO:

egosh ego restart

12.  Enable applications:

soamcontrol app enable <appName>

5.    Uninstallation

If required, follow the instructions in this section to uninstall this interim fix from your cluster.

1.       Log on to the master host as the cluster administrator, disable applications, and stop the following services:

egosh user logon -u Admin -x Admin

 soamcontrol app disable all

egosh service stop WEBGUI REST RS SD

2.       Log on to any management host in the cluster and restore the following files from your backup:

cd $EGO_TOP

tar -xvf backup.tar

3.       Copy the following configuration file to your shared directory:

cp $EGO_TOP/gui/conf/useracl/permission_GUIPermissionSoam.acl $EGO_CONFDIR/../../gui/conf/useracl/

4.       Open the $EGO_CONFDIR/../../gui/conf/pmcconf/pmc_conf_ego.xml file and remove the following sections:

<Parameter>
   <Name>RestrictHostLogRetrieve</Name>
      <!-- This parameter restricts retrieving logs for a host. -->
      <!-- Valid values are true or false. By default, this parameter is false, so that there is no restriction: all logs from any directory can be retrieved. -->
      <Value>true</Value>
</Parameter>
<Parameter>
    <Name>WhitelistLogsDir</Name>
       <!--This parameter takes effect only when the RestrictHostLogRetrieve parameter is set to true. -->
       <!--This parameter allows you to define the specific directories from which logs can be retrieved for a host. -->
       <!--Define any number of directories as required, separating multiple directories by a semicolon (";"). -->
       <Value>${SOAM_HOME};${EGO_TOP}</Value>
</Parameter>

5.       Restore the $EGO_CONFDIR/../../gui/conf/navigation/pmc_menu.xml

6.       Edit the $EGO_CONFDIR/ego.conf file to remove the EGO_ENABLE_UNAVAILABLE_HOST_IN_RG parameter.

7.       Clean up the gui/work directories by deleting all subdirectories and files from the following directories:

rm -rf $EGO_TOP/gui/work/*

rm -rf $EGO_TOP/gui/workarea/*

rm -rf $EGO_TOP/kernel/rest/workarea/*

rm -rf $WLP_OUTPUT_DIR/webgui_hostname/gui/workarea/*

8.       Start the WEBGUI, REST, RS and SD services:

egosh service start WEBGUI REST RS SD

9.       From the master host, restart EGO:

egosh ego restart

10.     Enable applications:

soamcontrol app enable <appName>

6.    List of files

gui/conf/useracl/permission_GUIPermissionSoam.acl

gui/3.6/lib/rest-ego-3.6.0.jar

gui/3.6/lib/soamgui.jar

gui/3.6/lib/egogui.jar

gui/3.6/lib/commons-ego.jar

gui/3.6/lib/newegogui.jar

wlp/usr/servers/gui/apps/perf/3.6/perfgui/WEB-INF/classes/com/platform/perf/report/action/ReportAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/wizard/CreateAppWizardOperator.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/application/RegisterAppXmlAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationRestResource.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationV1RestResource.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/action/applications/AddApplicationAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/action/appConf/ImportAppProAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/classes/com/platform/gui/pmr/web/action/appConf/ExportAppProfileAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/symgui/WEB-INF/lib/soamgui.jar

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/api/symping/SympingConfSetting.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/wizard/CreateAppWizardOperator.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/wizard/importAppProActionWizard.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/srvpackage/ExportDeploymentXmlAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/srvpackage/ConfigurePackageProfileAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/appConf/GetApplicationConfAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/appConf/ImportAppProAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/appConf/ExportAppProfileAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/common/FormatXMLAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/WEB-INF/classes/com/platform/soam/gui/web/application/RegisterAppXmlAction.class

wlp/usr/servers/gui/apps/soam/7.1.2/soamgui/symping/symping.jsp

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/wizard/CreateAppWizardOperator.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/application/RegisterAppXmlAction.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationRestResource.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationV1RestResource.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrestv1/WEB-INF/lib/soamgui.jar

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/wizard/CreateAppWizardOperator.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/general/mapreduce/gui/web/application/RegisterAppXmlAction.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationRestResource.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/classes/com/platform/gui/pmr/web/rest/ApplicationV1RestResource.class

wlp/usr/servers/rest/apps/soam/7.1.2/pmrrest/WEB-INF/lib/soamgui.jar

wlp/usr/shared/resources/rest/3.6/commons-ego.jar

3.6/linux-x86_64/bin/egosh

3.6/linux-x86_64/etc/rs

3.6/linux-x86_64/etc/vemkd

3.6/linux-x86_64/lib/jni/libVEMApiCommon.so

soam/7.1.2/linux-x86_64/bin/soamdeploy

soam/7.1.2/linux-x86_64/etc/sim

soam/7.1.2/linux-x86_64/etc/ssm

soam/7.1.2/linux-x86_64/etc/sd

Jupyter-4.1.0.tar.gz

7.    Copyright and trademark information

© Copyright IBM Corporation 2018

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM®, the IBM logo, and ibm.com® are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.