Readme file for IBM Spectrum Conductor with Spark 2.2.1 Interim Fix 504187

Readme file for: IBM® Spectrum Conductor with Spark
Product/Component Release: 2.2.1
Update Name: Interim Fix 504187
Fix ID: cws-2.2.1-build504187-jpmc

Publication date: 22 October 2018
Last modified date: 19 October 2018

Description

To enforce cluster level authentication and encryption settings, the following parameters are now supported in ascd.conf:

· CONDUCTOR_SPARK_ENFORCE_SPARK_EGO_AUTH_MODE

· CONDUCTOR_SPARK_ENFORCE_ENCRYPTION

· CONDUCTOR_SPARK_ENFORCE_SECURITY_SPARK_AUTH

· CONDUCTOR_SPARK_ENFORCE_NOTEBOOK_SSL

These parameters handle the cluster level enforcement of the authentication and encryption parameters which are normally available to users to select. Enforcement of these parameters results in the user still being able to see the GUI elements when they register or modify Spark instance groups, but they will not be able to change the options that are enforced.

About each parameter

CONDUCTOR_SPARK_ENFORCE_SPARK_EGO_AUTH_MODE

Enforce the Spark master to either authenticate and authorize, or to trust the specific submission user. If EGO_AUTH or EGO_TRUST is specified, users will not be able to choose by themselves during Spark instance group registration or modification. A value of "EGO_AUTH" forces authentication to be enabled. A value of "EGO_TRUST" will enforce authentication to be disabled, and all submission users will be trusted.

CONDUCTOR_SPARK_ENFORCE_ENCRYPTION

Enforce SSL encryption parameters in Spark. If a value is set, users are not able to choose by themselves during Spark instance group registration or modification. A value of "WORKLOADANDSPARKUIS" enforces SSL enablement for workload and Spark UIs (the master UI, driver UI, and history service UI) which correspond to the spark parameter spark.ssl.enabled=true. A value of "WORKLOADONLY" enforces SSL enablement for workloads only which corresponds to the Spark parameters spark.ssl.enabled=false, spark.ssl.ego.workload.enabled=true, and spark.ssl.ego.gui.enabled=false. A value of "SPARKUISONLY" enforces SSL enablement for Spark UIs only which corresponds to the spark parameters spark.ssl.enabled=false, spark.ssl.ego.workload.enabled=false, and spark.ssl.ego.gui.enabled=true. A value of "DISABLE" will enforce disable SSL which corresponds to the spark parameters spark.ssl.enabled=false, spark.ssl.ego.workload.enabled=false, and spark.ssl.ego.gui.enabled=false.

CONDUCTOR_SPARK_ENFORCE_SECURITY_SPARK_AUTH

Enforce Spark security authentication in Spark security parameters. If either TRUE or FALSE, users will not be able to choose by themselves during Spark instance group registration or modification. A value of "TRUE" enforces the spark parameter spark.authenticate, to be true. A value of "FALSE" enforces the parameter to be false.

CONDUCTOR_SPARK_ENFORCE_NOTEBOOK_SSL

Enforce notebook SSL. If either TRUE or FALSE, users are not able to choose by themselves during Spark instance group registration or modification. A value of "TRUE" enforces notebook SSL to be true. A value of "FALSE" enforces notebook SSL to be false.

1. List of fixes

2. Download location

3. Product notifications

4. Products or components affected

5. Installation and configuration

6. List of files

7. Copyright and trademark information

1. List of fixes

1. RFE 120489 – Enforce cluster level authentication and encryption settings

2. Download Location

Download Fix 504187 from the following location: http://www.ibm.com/eserver/support/fixes/

3. Product notifications

To receive information about product solution and patch updates automatically, subscribe to product notifications on the My Notifications page http://www.ibm.com/support/mynotifications/ on the IBM Support website (http://support.ibm.com). You can edit your subscription settings to choose the types of information you want to get notification about, for example, security bulletins, fixes, troubleshooting, and product enhancements or documentation changes.

4. Products or components affected

Affected components include: ascd, conductorspark_gui

5. Installation and configuration

Before installation

For Linux and Unix:

1. Create a backup directory and back up the following files:

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/i18n/locale-en.json

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/js/configureInstallation.controller.js

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/js/configureNotebook.controller.js

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/js/instance.configure.controller.js

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/js/sigTemplate.configure.controller.js

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/configureInstallation.html

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/configureNotebook.html

· $EGO_TOP/wlp/usr/servers/gui/apps/conductor/2.2.1/conductorgui/spark/instance/instanceConfigureBasicSettings.html

· $EGO_TOP/ascd/2.2.1/lib/asc-common-2.2.1.jar

· $EGO_TOP/ascd/2.2.1/lib/asc-core-2.2.1.jar

· $EGO_CONFDIR/../../ascd/conf/ascd.conf

Installation steps

1. Log into the management host as CLUSTERADMIN and source the environment.

2. Stop the ascd service.

egosh service stop ascd

3. Move the patch tar file into $EGO_TOP, and then untar the patch tar file.

tar zoxf cws-2.2.1.0_x86_64_build504187.tgz -C $EGO_TOP

4. Configure the new supported parameters in $EGO_CONFDIR/../../ascd/conf/ascd.conf by adding the following lines to the file and uncommenting the desired parameters:

###################################

#Enforce specific security settings

###################################

#Enforce the Spark master to either authenticate and authorize, or to trust the specific submission user.

#If either EGO_AUTH or EGO_TRUST, users will not be able to choose by themselves during Spark instance group registration or modification.

#A value of "EGO_AUTH" will force authentication to be enabled. A value of "EGO_TRUST" will enforce authentication to be disabled, and all submission users will be trusted.

#CONDUCTOR_SPARK_ENFORCE_SPARK_EGO_AUTH_MODE=EGO_AUTH

#Enforce SSL encryption parameters in Spark

#If a value is set, users will not be able to choose by themselves during Spark instance group registration or modification.

#A value of "WORKLOADANDSPARKUIS" - Enforce enable SSL for workload and Spark UIs (the master UI, driver UI, and history service UI)

#A value of "WORKLOADONLY" - Enforce enable SSL for workload only

#A value of "SPARKUISONLY" - Enforce enable SSL for Spark UIs only

#A value of "DISABLE" - Enforce disable SSL

#CONDUCTOR_SPARK_ENFORCE_ENCRYPTION=WORKLOADANDSPARKUIS

#Enforce Spark security authentication and SASL encryption in Spark Security parameters

#If either TRUE or FALSE, users will not be able to choose by themselves during Spark instance group registration or modification.

#A value of "TRUE" will enforce spark.authenticate to be true.

#A value of "FALSE" will enforce that parameter to be false.

#CONDUCTOR_SPARK_ENFORCE_SECURITY_SPARK_AUTH=TRUE

#Enforce Notebook SSL

#If either TRUE or FALSE, users will not be able to choose by themselves during Spark instance group registration or modification.

#A value of "TRUE" will enforce notebook SSL to be true.

#A value of "FALSE" will enforce notebook SSL to be false.

#CONDUCTOR_SPARK_ENFORCE_NOTEBOOK_SSL=TRUE

Note: If you previously had the CONDUCTOR_SPARK_ENFORCE_SECURITY_SPARK_AUTH_AND_SASL_ENCRYPT parameter in ascd.conf, it is now obsolete and should be removed.

5. Start the ascd service.

egosh service start ascd

6. Clear your browser cache and log into the cluster management console as usual.

Uninstallation

1. Log in to the management host as CLUSTERADMIN and source the environment.

2. Stop the ascd service.

egosh service stop ascd

3. Replace the files from the backup directory created at the beginning of the process.

4. Start the ascd service.

egosh service start ascd

5. Clear your browser cache and log in to the cluster management console.