Readme File for Installing IBM Spectrum Conductor Deep Learning Impact 1.1 with PowerAI 1.5.1 on POWER9

Readme file for: IBM Spectrum Conductor Deep Learning Impact

Product/Component Release: 1.1.0

Fix ID: dli-1.1.0-build489874

Publication date: April 27, 2018

 

With this iFix, you can install IBM Spectrum Conductor Deep Learning Impact 1.1 with PowerAI 1.5.1 on POWER9. For more details on PowerAI 1.5.1, refer to the PowerAI website at https://www.ibm.com/us-en/marketplace/deep-learning-platform/resources.

 

1.      Packages

2.      System requirements

3.      Installation

1.      Prerequisites

2.      Preparing to install IBM Spectrum Conductor Deep Learning Impact

3.      Installing IBM Spectrum Conductor Deep Learning Impact and applying iFix 489874

4.      Configuring IBM Spectrum Conductor Deep Learning Impact

4.      Get started

5.      Known issues

6.      Limitations

7.      More information

8.      Copyright and trademark information

 

1. Packages

 

Use the following packages to install IBM Spectrum Conductor Deep Learning Impact 1.1 with PowerAI 1.5.1 on POWER9:

 

Packages

dli-1.1.0-build489874.tar.gz

 

Obtain the iFix package from IBM Fix Central: www.ibm.com/support/fixcentral

cws-2.2.1.0_ppc64le.bin

dli-1.1.0.0_ppc64le.bin

cws_entitlement.dat

dli_entitlement.dat

Obtain IBM Spectrum Conductor Deep Learning Impact 1.1 product files from IBM Passport Advantage: www.ibm.com/software/howtobuy/passportadvantage

 

 

 

2. System requirements

 

Hardware requirements

 

Before installing IBM Spectrum Deep Learning Impact 1.1.0 with PowerAI 1.5.1 on POWER9, you must satisfy all the PowerAI 1.5.1 hardware requirements, refer to the PowerAI 1.5.1 readme found here: /opt/DL/README.md.

 

Software requirements

 

Before installing IBM Spectrum Deep Learning Impact 1.1.0 with PowerAI 1.5.1 on POWER9, you must satisfy the software requirements, as follows:

 

Software requirements for IBM Spectrum Conductor Deep Learning Impact on POWER9

Operating system

Red Hat Enterprise Linux (RHEL) 7.5 + Security fix: RHSA-2018:1374 available at https://access.redhat.com/errata/RHSA-2018:1374

IBM PowerAI

 1.5.1

 

IBM Spectrum Conductor with Spark

2.2.1 + latest fixes from IBM Fix Central

 

 

3. Installation

 

3.1 Prerequisites

 

Before installing IBM Spectrum Conductor Deep Learning Impact, you must:

 

3.2 Preparing to install IBM Spectrum Conductor Deep Learning Impact

 

First, you must prepare for installing IBM Spectrum Conductor Deep Learning Impact by installing IBM PowerAI and dependency packages on a dedicated Anaconda environment for deep learning. For instructions on how to install IBM PowerAI and dependency packages on a dedicated Anaconda environment for deep learning, see Preparing to install IBM Spectrum Conductor Deep Learning Impact at http://www.ibm.com/support/knowledgecenter/en/SSWQ2D_1.1.0/in/installation-prepare-POWER9-master.html.

 

After installing IBM Spectrum Conductor with Spark 2.2.1, you must apply the latest fixes from IBM Fix Central.

 

 

3.3 Installing IBM Spectrum Conductor Deep Learning Impact and applying iFix 489874

Once your IBM Spectrum Conductor with Spark environment is installed along with the latest fixes, you can install and configure the IBM Spectrum Conductor Deep Learning Impact add-on package. To install IBM Spectrum Conductor Deep Learning Impact see, Installing IBM Spectrum Conductor Deep Learning Impact at www.ibm.com/support/knowledgecenter/en/SSWQ2D_1.1.0/in/installation-dli-master-host-install.html.

After installing IBM Spectrum Conductor Deep Learning Impact, apply the iFix package (dli-1.1.0-build489874.tar.gz) for IBM Spectrum Conductor Deep Learning Impact to work with PowerAI 1.5.1 on POWER9.

 

1.     Make sure that there are no running applications and that the cluster is shutdown. Log into the master host as root, and take the following steps to shutdown the cluster. Note, that we are using /opt/ibm/spectrumcomputing as the top installation directory.

a.      Source the environment.

# . /opt/ibm/spectrumcomputing/profile.platform

 

b.      Log in as a cluster administrator.

# egosh user logon -u Admin -x Admin

 

c.      Stop all cluster services.

# egosh service stop all

 

d.      Ensure all services are in DEFINED state.

 

# egosh service list

 

e.      Shutdown the cluster.


# egosh ego shutdown all

 

2.      Log in to the master host as root.

3.      Download the iFix package (dli-1.1.0-build489874.tar.gz) from IBM Fix Central to a local directory on the master host, for example: /iFix_489874

4.      Change directory to DLI_SHARED_FS and back up the fabric directory. For example, assuming DLI_SHARED_FS is /dli_shared:


# cd /dli_shared
# mv fabric fabric.bak

 

5.      Untar the package dli-1.1.0-build489874.tar.gz. Here, egoadmin is your IBM Spectrum Conductor with Spark cluster administrator operating system user.

# mkdir -p /dli_shared/fabric && tar zxf /iFix_483732/dli-1.1.0-build489874.tar.gz -C /dli_shared/fabric
# chown -R egoadmin:egoadmin /dli_shared/fabric

6.      Start the cluster.


# egosh ego start

 

 

3.4 Configuring IBM Spectrum Conductor Deep Learning Impact

You must configure IBM Spectrum Conductor Deep Learning Impact before you can start using it. To configure IBM Spectrum Conductor Deep Learning Impact, see Configuring IBM Spectrum Conductor Deep Learning Impact after installation at www.ibm.com/support/knowledgecenter/en/SSWQ2D_1.1.0/in/installation-dli-configuration.html.

 

4. Get started

To get stated with IBM Spectrum Conductor Deep Learning Impact 1.1.0:

·        Review the release notes to familiarize yourself with the details of this release:

www.ibm.com/support/knowledgecenter/en/SSWQ2D_1.1.0/gs/dli_release_notes.html

·        Review other topics in the “Getting Started” section to better understand IBM Spectrum Deep Learning Impact:

www.ibm.com/support/knowledgecenter/en/SSWQ2D_1.1.0/gs/getting-started.html

·        After you install and configure the product, use the topics in the “Spark workload” section to help you start using IBM Spectrum Conductor Deep Learning Impact.

www.ibm.com/support/knowledgecenter/en/SSZU2E_2.2.1/managing_cluster/spark_workload_container.html

·        Samples are available on IBM Cloud.

https://git.ng.bluemix.net/ibmcws-deep-learning-samples/ibmcws-deep-learning-tensorflow-samples

 

5. Known issues

IBM JDK 8.0.0 does not support IBM POWER9, as a result, after installing IBM Spectrum Conductor Deep Learning Impact, download and install the correct version of Java, using the following steps:

1.      Log into the master host as root.

2.      Download the IBM Java SDK 8.0.4.11 bin file from http://public.dhe.ibm.com/ibmdl/export/pub/systems/cloud/runtimes/java and copy it to the master host local drive. For example:

/root/ibm-java-ppc64le-sdk-8.0-4.11.bin

Note: If you encounter any issues obtaining this file, contact IBM Support.

3.      Change the bin file to executable. For example:

# chmod +x ibm-java-ppc64le-sdk-8.0-4.11.bin

4.      Run and install IBM Java and follow the installation wizard to install it to the default location (/opt/ibm/java-ppc64le-80).

./ibm-java-ppc64le-sdk-8.0-4.11.bin

5.      Shutdown the cluster. Skip this step if the cluster is not started yet.

Note: $EGO_TOP is your IBM Spectrum Conductor with Spark installation top directory, for example: /opt/ibm/spectrumcomputing.

# . $EGO_TOP/profile.platform

# egosh user logon -u Admin -x Admin

# egosh service stop all

# egosh ego shutdown all

6.      Link cluster JRE to IBM SDK 8.0.4.11 with the following steps:

# cd $EGO_TOP/jre/3.6

# mv linux-ppc64le linux-ppc64le.bak

# ln -s /opt/ibm/java-ppc64le-80 linux-ppc64le

7.      Start the cluster.

# egosh ego start

 

6. Limitations

The following limitations apply to IBM Spectrum Deep Learning Impact 1.1.0 with PowerAI 1.5.1 on POWER9:

·        No high availability support on POWER9

 

 

7. More information

To obtain more information about IBM Spectrum Conductor Deep Learning Impact, see IBM Knowledge Center at www.ibm.com/support/knowledgecenter/en/SSWQ2D_1.1.0.

For any questions regarding this iFix, ask us directly on our Slack channel. For instructions on how to sign up for our Slack channel, see www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Wa0adcb6b782c_4e8b_b3c8_d633cb9456d8/page/Slack%20channel%20(IBM%20Cloud%20Technology)%20sign%20up%20page  

 

8. Copyright and trademark information

© Copyright IBM Corporation 1992, 2018.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM®, the IBM logo and ibm.com® are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.