Readme file for IBM® Spectrum Conductor 2.5.0 Interim Fix 600720
Readme file for: IBM Spectrum Conductor
Product/Component release: 2.5.0
Update name: Interim Fix 600720
Fix ID: sc-2.5-build600720-jpmc
Publication date: September 28, 2021
Interim
fix to enhance Spark 3.0.1 to load data from JDBC data sources with Kerberos
authentication.
1. List of fixes
2. Download location
3. Product and components affected
4. Installation and configuration
5. List of files
6. Product notifications
7. Copyright and trademark information
APAR: P104380
Download interim fix 600720 from the following location: http://www.ibm.com/eserver/support/fixes/.
Component name, Platform, Fix ID:
Spark 3.0.1, linux-x86_64 or linux-ppc64le, sc-2.5-build600720-jpmc
1. Download the sc-2.5.0.0_build600720.tgz package to a local directory on your computer. Decompress the file. Once decompressed, you will have the following Spark packages:
Spark3.0.1-Conductor2.5.0.tgz
2. Launch a browser and log in to the cluster management
console as a cluster administrator.
3. Remove the existing Spark package from the cluster. For example, to remove the existing Spark 3.0.1 package:
a. Click Resources > Frameworks > Spark Management.
b. Select version 3.0.1.
c. Click Remove.
d. In the confirmation dialog, click Remove.
4. Add the new Spark package to the cluster. For example, to add the new Spark 3.0.1 package:
a. Click Resources > Frameworks > Spark Management.
b. Click Add.
c. Click Browse and select the Spark3.0.1-Conductor2.5.0.tgz package downloaded previously.
d. Click Add.
5. Download the Oracle JDBC
(ojdbc8.jar) from https://www.oracle.com/database/technologies/appdev/jdbc-ucp-21-1-c-downloads.html. You will specify it to the ${SPARK_HOME}/jars directory when configuring Spark instance group in step 7.
6. Create
your IBM Spectrum Conductor instance group using the cluster management console.
7. Set
Spark configuration for the instance group:
a. From Workload > Instance Groups, select the instance group, and click Configure.
b. In the Spark tab, click Configuration to configure Spark.
c.
Set the Oracle
JDK location (For example: JAVA_HOME=/opt/jdk1.8.0_301) for your instance group.
d. Go to Additional Parameters, click Add
a Parameter, and set:
·
Parameter:
spark.hadoop.hadoop.security.authentication
·
Value: kerberos
e. Click
Save.
f. Switch
to Packages tab, click Create Single-File Packages, select the ojdbc8.jar
package downloaded.
g. Click
Modify Instance Group.
8.
In the Jupyter
user interface cells, access a Kerberos-enabled Oracle database, by adding the principal and keytab parameters. For example:
df_table = spark.read.format("jdbc")
\
.option("url","jdbc:oracle:thin:@9.21.53.194:1521:ORCLCDB") \
.option("dbtable",
"test01") \
.option("principal","testuser1@PLATFORM.COM") \
.option("keytab", "/tmp/testuser1.keytab") \
.load()
9. Create
a Spark SQL temporary view and show the results. For example:
df_table.createOrReplaceTempView("test01")
tempDF = spark.sql("select *
from test01;")
tempDF.show()
Spark3.0.1-Conductor2.5.0.tgz
To receive information about
product solution and patch updates automatically, subscribe to product
notifications on the My Notifications page
http://www.ibm.com/support/mynotifications/ on the IBM Support website (http://support.ibm.com). You can edit your subscription settings to choose the types of information you want to get notification about, for example, security bulletins, fixes, troubleshooting, and product enhancements or documentation changes.
© Copyright IBM Corporation 2021
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
IBM®, the IBM logo and ibm.com® are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml