IBM Fluid Query v1.7.1 Readme ===================================================== For the instructions to install and use the IBM Fluid Query software, see the IBM Fluid Query User Guide (IBM_fluid_query_user_guide.pdf), or the online documentation that is available in IBM Knowledge Center (https://www.ibm.com/support/knowledgecenter/SSULQD_1.7.1/com.ibm.nz.hdp.doc/kc_welcome_fq.html). You can obtain the software and documentation at IBM Fix Central (http://www-933.ibm.com/support/fixcentral/). You must have a customer account that allows you obtain the IBM Netezza software downloads. Changes in this release -------------------------- v1.7.1 Includes the following improvements: * You can now automatically create views on NPS of the data that you import to Hadoop. The functionality is called automatic federation. * Hive partitioning and clustering is now supported when moving data from NPS. * You can now export a Hive table that was not previously imported from NPS. * You can now use checksum calculation to verify data integrity and consistency after the data movement process. * The process of data movement from Hadoop was improved with the fdm.sh script. You no longer have to use nzcodec.jar and export the Hadoop classpath. * The so called "local mode" of data movement, when IBM Fluid Query connects to Hive directly in order to create table metadata, is now deprecated. Remote mode is used instead by default, that is IBM Fluid Query uses a JDBC driver to connect to Hive or BigSQL. This approach reduces a number of limitations. * Security was enhanced with the ability to encrypt passwords that are stored in connector configuration files using the 128-bit AES key. * The installation procedure was modified: You are asked to provide the paths to Hadoop /lib and /conf files during installation, to reduce additional configuration and simplify productive use of fast data movement. * IBM Fluid Query has been certified for the following distributions: Big SQL 4.2, Hortonworks 2.5, Cloudera Impala 5.9, Spark 1.6.1 and DB2 LUE v11.1. v1.7.0.1 Includes the following improvements and defect fixes: * You can now use multi-byte character set support for table column names. * NPS data type DATE can now mapped to Hive DATE instead of STRING for Hive versions 0.12 and above. * Fixed: Fluid Query Data Movement sends NULL for the whole column with timestamp containing value 00:00:00. * Fixed: Microseconds part of timestamp is truncated when using Oracle connector. * Fixed: Generic connector templates are not installed in the upgrade scenario. v1.7 The following improvements were introduced: * Online documentation for IBM Fluid Query 1.7 is available in IBM Knowledge Center at https://www.ibm.com/support/knowledgecenter/. Documentation is enhanced with a 'Getting started' section including best practices, and tutorials. Online documentation is updated regularly to provide up-to-date information on the product. * You can now transfer data from NPS to Hadoop to Hadoop-specific formats: PARQUET, AVRO, ORC, RCFILE. * Improvements in the process of importing schema and tables with nz.fq.tables property. * Support for Cloudera 5.5.1. * Validation of configuration before running data movement can now be executed on Hadoop. * New configuration templates are available to facilitate generic connector configuration. * Kerberos authentication is supported with a generic connector. * Generic connector provides a password prompt if the password is not provided in the command line. The password does not have to be hardcoded in the configuration XML file any more. v1.6.0.1 Includes several improvements: * Fixed defect regarding an unexpected behavior of import on full-schema enabled system if schema name is not provided. * Fixed defect regarding hanging queries when multiple connector queries are executed in parallel. * Introduced enhancement for handling a field delimiter and end of line character in data movement: During decompression from the NPS internal format, newline and carriage return characters are replaced with a space (' '). This is performed by the nzetc tool only when querying binary format. Mixed mode is not affected. v1.6 Includes several fixes and updates including the following changes: * IBM Netezza Analytics 3.2.1 is now supported, and IBM Fluid Query supports Java 7 JDBC drivers with this version of IBM Netezza Analytics. * Generic connector is now available to allow users to connect IBM Fluid Query to any provider using a JDBC driver. * The append mode in data movement feature now allows for incremental import, that is, any data that was added after last data movement is appended to the table. * The nz.fq.tables property of the fq-import-conf.xml configuration file now supports the asterisk (*) character to import all of the tables in the specified database. * During data import, the import process now checks for unsupported object types like views, external tables, row-secure tables, and materialized views and displays an error if any unsupported object is present. * Remote mode data connections now support the ability to use multiple Hadoop services and versions as the same time. Drivers for DB2, dashDB, PDoA and BigSQL can now be used at the same time, and drivers for Hive, Spark SQL, Impala, and BigSQLv1 can be used at the same time. * The data connector feature now supports the ability to read national characters in the records obtained from the remote service provider. v1.5.0.1 Includes several fixes and updates including the following changes: * The remote mode of the data connector feature is no longer limited to a single service. * Adds support for the Cloudera JDBC drivers. * Fixes a problem with the export of tables that were imported using uppercase names. * Fixes a performance issue where the FqRead transferred records from the remote system when only metadata was needed. * Fixes an issue where user-defined table functions failed when registered in version 1.5 using a configuration properties file created in version 1.0. * Assorted usability and documentation updates. v1.5 * In addition to Hadoop systems, IBM Fluid Query can now be used to connect an NPS system with the following services: – Another IBM PureData® System for Analytics system – IBM DB2 – IBM dashDB – IBM PureData System for Operational Analytics – Apache Spark SQL – Oracle These connections are limited to only the data connector feature of IBM Fluid Query. Fast data movement is not supported for them. * You can now run fast data movement from an IBM Netezza system or any other system that runs Java. * The installation script is renamed to fluidquery_install.sh. It no longer requires the --datamove parameter when installing fast data movement. 97010 - Adds support for using Hive to query compressed (binary) files on Hadoop after they have been imported in this format using the data movement feature. 99177 - For Impala service only, adds an fqConfigure.sh --no-auth option to use no authentication (auth=noSasl) for the connection. v1.0.0.1 * Adds support for IBM BigInsights 4.0 as a supported Hadoop environment. * Updates the user guide to note Hive complex data types that are not supported with the data connector. * Adds a -version switch to display the version of the data movement software installed on the Hadoop system using the command: hadoop jar /fluidqueryLocal/nzcodec.jar -version. =============================================================================== Copyright(c) 2017. IBM Corporation. All rights reserved.