Readme and Release notes for release 4.1.0.6 General Parallel File System Express Edition 4.1.0.6 GPFS_EXP-4.1.0.6-power-AIX Readme

Unless specifically noted otherwise, this history of problems fixed for GPFS 4.1.x applies for all supported platforms.

Problems fixed in GPFS 4.1.0.6 [February 5, 2015]

• Customer can change the spares of an existing DA when adding new pdisk.
• Update code to prevent a deadlock that could occur when multiple threads try to create the same file in a directory at same time.
• When E_NOMEM occurs on a token manager during token transfer, try throw out half of all OpenFile objects, regardless of which file system they belong to. Also limit implicitly set maxStatCache(MSC) value to 10,000.
• Reduce memory utilization for GPFS RDMA QPs and fix a problem with Connect-IB when verbsRdmaSend is enabled.
• Fixed the assert by mapping the error to permission error as expected.
• Fixed a problem where creating a clone copy file, in an encrypted file system may result in threads being blocked or in an abnormal daemon termination.
• Correct tsbuhelper updateshadow command to recognize several circumstances when gencount changes even for files not yet backed up, or changed.
• Fixed the problem by changing the implementation of flushpending while operations get requeued.
• In FPO, use disks evenly when doing mmrestripefs -r or mmdeldisk.
• Fix is especially recommended for GSS systems on the Power platform.
• Fixed the problem by allowing prefetch recovery to run only if all the nodes in the cluster are all at at least minReleaselevel 1410.
• Fixed the problem by clearing CA_STATE|CA_CREATE bits after local rm is done.
• Fix a defect which may cause data consistency problem if one runs mmrestripefile after reducing replica level.
• Fix is recommended for all GSS systems using NVRAM log tip devices.
• When creating FPO file system, always use 'cluster' layoutMap if allowWriteAffinity is yes and layoutMap is unspecified.
• Fixed a problem that enable file system quota would hit an assert.
• "Waiting for nn user(s) of shared segment" messages on shutdown.
• Fixed the problem by reading inmem attrs for dump command instead of reading it from disk.
• Fsck reports false corruption in inode alloc map due to rounding on wrong sector size.
• Fix code that could cause mmclone copy to fail incorrectly with EINVAL. This problem could only occur when source is file in GPFS snapshot.
• Fix GNR bug related to a race condition that cause recovery failure during startup or failover.
• Fix a mmdeldisk problem caused by disabled quota files placed in the system pool.
• Fixes a problem where fsck hits signal 8 during inode validation
• Make sure that the inode pointed to by different dentry is not the same inode to prevent possible deadlock by 2 diferent dentries pointing to the same inode and trying to lock the same inode twice.
• Fix potential loss of IO error in linux io_getevents() call when enableLinuxReplicatedAio is enabled (3.5.0.14+). Fix a problem that returns 'error 217' improperly when do Direct IO on a replicated file which haspartial replicas on unavailable disks (4.1.0+).
• Fix a stack corruption issue.
• Fix a linux lookup crash issue.
• Apply if secrecy of file metadata (pathnames, attributes and extended attributes) is a concern.
• Revised disk selection algorithm to ensure no reuse can exist in a map entry being modified by rebuild or rebalance. Prior algorithm allowed this type of reuse and could lead to lessening the failure tolerance, trailer validation errors, and assert crashes in various GSS subsystems.
• Ensure that GPFS uses only secure CBC padding schemes when exchanging data over TLS. This affects customers who have set cipherList to a real cipher (i.e. not AUTHONLY) in order to authenticate and encrypt data passed between nodes/clusters. This also affects customers who use file-level encryption.
• Fix GPFS_FCNTL_GET_DATABLKLOC API, make it returns correct disk ID for data in inode files.
• This fix is required for GSS server nodes in which multiple names assigned to the node differ only in the domain portion of the name.
• Correct a problem leading to misleading tsgskkm system clock complaints.
• Fix is recommended in all GNR configurations.
• This update addresses the following APARs: IV67901 IV68007 IV68059 IV68060 IV68062 IV68064 IV68065 IV68096 IV68491 IV68493.

Problems fixed in GPFS 4.1.0.5 [December 5, 2014]

• Fix an alloc cursor issue in block allocation code that may lead to spurious no space error in FPO file system.
• Fixed code to consider flags during a disks challenge of the current cluster manager.
• This fix applies to GNR/GSS customers that are adding additional server nodes and cluster is created using default block size.
• Reduce number of nsdMsgRdmaPrepare messages sent.
• Fix GSS bug related to concurrent overlapping read operations during failover/error scenarios.
• Fixed a problem that the range size was initialized to wrong value with meta data-block size instead of the correct data block size.
• Fix the problem that for certain configuration of FPO cluster, replicas failed to involve all LGs.
• Redirect automatic recovery's tsrestripefs output to /var/adm/ras/restripefsOnDiskFailure.log
• fix problem with verbsRdmasPer[Node Connection] set to a value of 1.
• Reduce CNFS failover time on systems with large list of exports.
• Allow disk addresses in inode 5 (Extended Attribute File) be be found by the mmfileid command.
• Fix GSS bug related to mixed read-write workload with a working set size that matches GSS cache memory size.
• Fixed a problem that range number was initialized to wrong value in a different meta data-block size environment anda full block write after lseek without placement installed.
• fcntl revokes referencing a completed/freed request.
• Hadoop File System API open() in connector throws exception as hdfs when user has no permission to access a file.
• Fix bug where mmsetquota set the inode limit to unlimited if only changeblock quota is requested and vice versa.
• If the user of a GSS system had previously changed the slow disk detection parameters manually to the following values: nsdRAIDDiskPerformanceMinLimitPct=50 and nsdRAIDDiskPerformanceShortTimeConstant=25000, then they can now remove the manual setting; but they don't have to remove it.
• Fixed problem where the gpfs daemon gets sig11 when application calls GPFS_FCNTL_GET_DATABLKLOC api in mixed PPC64/X64 cluster.
• This update addresses the following APARs: IV66617 IV66620 IV66621 IV67005 IV67006.

Problems fixed in GPFS 4.1.0.4 [October 31, 2014]

• Fix code to prevent a GPFS daemon assert that could occur after automatic remount of filesystems. The problem only occurs on Linux node when user issued GPFS commands to access the filesystem before automatic remount has completed.
• Fix "subscript out of range" error in recoverFailedDisk.
• Correct --single-instance option for mmapplypolicy runs against directory.
• The fix avoids a bug where pdisks become "missing" due to descriptors being overwritten. The fix is recommended if system firmware upgrades are applied.
• Fix code used to check user provided buffer for NULL where it can cause writev() to incorrectly fail with EINVAL. Both readv()/writev() could be affected.
• Fixed the problem that O_DSYNC was not honored on data write to small files.
• Fixed Ganesha thread deadlock caused by Ganesha up-call thread's getting byte range lock that was just released.
• Fix ibv_modify_qp error 22 when RDMA connecting client mlx4_0 port 2 HCA to server single port qib0 HCA.
• Fix rare race condition that could result in a deadlock when moving data from the inode into a regular data block.
• Too many RDMA connection created between nodes.
• Fix Signal 11 on Connect-IB.
• Make sure that FHs are still pointing to directories after we get the lock for rename.
• 1) Make sure stop Ganesha as soon as GPFS daemon cleanup starts 2) Make sure that FHs are still pointing to directories after we get the lock for rename.
• Fix problem where AFM does not correctly replay Asynchronous I/O (AIO) writes, such as with aio_write(), when the file is opened with O_DIRECT flag. The problem may cause files to exhibit inconsistency between cache and home.
• Fixed a timing issue which may cause AFM to miss replaying updates to home, if the update happens after a period of long inactivity. Disable a three-phase protocol during which requests are dropped while the gateway nodes are temporarily made inactive so they can be marked clean.
• This update addresses the following APARs: IV65497.

Problems fixed in GPFS 4.1.0.3 [September 18, 2014]

• The patch fixed mmfsd crash problem when restriping FS and adding disks running at the same time.
• Fixes a problem may cause mmchdisk start to hang if there are too many pools or diskless nodes in a FPO cluster.
• Fix a bug in mmdeldisk/mmadddisk that may cause file system to become unknown to GPFS if the name of that file system contains special char.
• Fixed the daemon assert by allowing the async reads as dependents on lookup.
• Fix local read only cache code that could caused GPFS daemon assert during file system unmount when local read only cache is in use.
• Fixed a possible cause of deadlock when mmpmon or the GPFS SNMP subagent does node registration.
• Improve handling and reporting of certain types of corrupted directory blocks.
• Fix a defect in the fileset snapshot restore tool that it may cannot restore a file properly when the file's link count is zero but there is still open instance for the file.
• mmlsfileset command used to display expiration timeouts for NON-RO fileset so if there any scripts which was looking for this will break.
• Fix code that can cause GPFS daemon to assert when running mkdir on an AFM/SECVM enabled filesystem with metadata replication. The problem will only occur when there is no AIX node and GPFS can't allocate disk space for both replica.
• Fix a truncate() issue. When truncate to a position which is beyond end of file on a FPO file system, the filesize could be set wrongly.
• Apply if you are troubled by mmapplypolicy not terminating promptly and cleanly when ENOSPC is encountered.
• Fix an assertion during mmrestoreconfig caused by quota file migration to reserved file.
• Fixed a rare problem in background deletion code due to uninitialized list pointers.
• Callback/user exit support is added for new event "afmFilesetUnmounted" which gets called when fileset is unmounted.
• prevent the GPFS daemon from running into the assertion when a GSS vdisk client IO fails.
• Fix race condition when renaming files.
• Reduce the kernel stack usage in the write code path to void potentional kernal stack overflow.
• This fix improves the GPFS tool useability to use mmaddcallback command to constantly monitor a desired file system status.
• Avoid possible core dump by protecting pclose() call if broken pipe is known. Also, add LCL_ESTALE to list of fatal conditions.
• This fix will detect the failure and report that the mmchfs command has failed.The FS will retain its old name and still be usable.
• fixed a memory overwritten problem caused by uninitialized string copying in mmfs_dm_query_session().
• Update restripe code to better handle both replica in same failure group after disk usage and failure group were changed via mmchdisk.
• Fix is recommended on all systems in which the disk enclosure supports slot power control. Currently, this includes only the P7IH.
• Prevent mmimgbackup from accepting --image option with a value that begins with an absolute path name such as /gpfs/ as this option is meant to only permit a file base name or prefix. Not for specifying the output directory.
• Fixed daemon failure on a cluster not configured with the ClusterConfiguration Repository (CCR), but which include Tie-Breaker disks which havea physical sector size of 4096 bytes.
• Fix AIO code path to properly manage vfs user count. Incorrect vfs user count could prevent file system quiesce and cause some GPFS commands (ex. snapshot commands) to fail.
• Improve handling of unexpectedly corrupted directory blocks.
• This fix is recommended for GPFS Native RAID systems running on the AIX operating system. It has no effect on Linux.
• Fix inode dealloc issue in AFM LU mode.
• Fix node failure recovery code to prevent a possible GPFS daemon assert. This problem could occur if file system manager node fails in the middle of restripe.
• Fixed the startup code which will handle the locks in previous uncleaned mmshutdown better.
• during mmcrfs, the client doesn't correctly parse the reply for 'nsdMsgConvertFormat', which causes the client failing to move on to the next backup nsd server.
• Avoid daemon assert when rejecting illegal mmcrsnapshot commands.
• Fix mmfsck assert "More than 22 minutes searching for a free buffer in the pagepool"
• fcntl revokes may block unmount under appropriate loads
• Fix a problem introduced by the fix for APAR IV56220 that will cause NSD server hangs/failures or possibly data corruption.
• Do not log fsstruct errors when offline fsck is running.
• When performing automatic disk recovery, take into account the value of defaultHelperNodes when initiating file system restripe operations.
• Add additional checks to ensure the uniqueness of certain node attributes.
• Fixed assert in setDataValid becuase blockOffset larger than the start offset.
• Fix repair code to always performance rebalance for none FPO file system as requested by user.
• Prevented I/O from being started after a node has lost contact with the cluster.
• Fix bug introduced in GPFS 3.4 PTF30 and GPFS 3.5 PTF20 where mmlsnsd -X doesn't display persistent reserve information of the disk.
• Fix code used to check user provided buffer for NULL where it can cause writev() to incorrectly fail with EINVAL. Both readv()/writev() could be affected.
• This update addresses the following APARs: IV63359 IV63464 IV63467 IV63469 IV63471 IV63516 IV63518 IV63824 IV63878 IV63882 IV63887 IV63889 IV63894 IV63901 IV63903 IV63911 IV63917 IV63927 IV64598 IV64862.

Problems fixed in GPFS 4.1.0.2 [August 4, 2014]

• Add tsm server selection option and improve messages.
• write(fd,NULL,bs) gives rc -1 and inconsistent behavior Added a check in code to validate if user provided buffer is NULL. If user provided buffer for rea/write system call is NULL than error is returned much earlier in code.
• Fix various problems with RDMA reconnection.
• Fix a rare case live lock which can happen when FPO file system is in low space situation.
• Fix two integer overflow problems of GPFS block map allocation module which caused by adding larger disk into existing file system. The problem can lead to block lost and data corruption.
• Avoid very rare log recovery failure after restripe of snapshot files.
• Prevent GPFS file system program mmfsd crash on a GNR/GSS system while deleting a log vdisk.
• Fix a problem in locking code that can cause GPFS daemon assert under certain rare race condition. The chance is slightly higher under 4.1.
• Prevent file system errors in the face of too many disk errors.
• Offline fsck fileset 0 record false positive on v3.3 and older filesystem.
• Fix a defect in the fileset snapshot restore tool when the tool tries to restore attributes of directories which they have been deleted after we create fileset snapshot.
• Apply if you see tsapolicy failing immediately after a helper node goes down.
• Eliminate FSSTRUCT errors from occuring during image restore process. Prevent gpfsLookup() common function from proceeding if stripe group is being image restored.
• Fix a node crash defect in gpfs_ireaddirx GPFS API when we use it to list changed dentry for a directory which has data in inode.
• Improved stability of mmfsadm dump tscomm.
• Install this fix if you have non-standard enclosure / drawer hardware not found in GSS systems.
• Fix a defect in the fileset snapshot restore tool when it tries to restore clone file which has been deleted after we create snapshot.
• Ignore Grace msg on nodes that do not support Ganesha.
• Fixed hung problem due ro lock overflow.
• Fix a problem where user registered callback is unexpectedly invoked when using mount instesad of mmmount.
• Fix a generation number mismatch defect when we create fileset in GPFS secvm file system.
• Fixed a race condition that could lead to an assertion failure when mmpmon is used.
• Fixed Assert 'filesetP->hasInodeSpace == 0' in offline fsck.
• Fixed problem in multi acquire and release with FGDL.
• When there is a GPFS failure return EUNATCH to Ganesha.
• Fileset snapshot restore tool restores dir attributes more effectively.
• Without this fix a setup with 4 or more drawers in an enclosure may not be able to survive the loss of the enclosure even though mmlsrecoverygruop -L states that disk group fault tolerance can survive an enclosure loss.
• Fix a defect that the restore tool cannot sync the restoring fileset when the file system manager node of the restoring fileset is running in GPFS 4.1.0.0 and the restore command is running in a node which runs upper version.
• Fixed online fsck assert after block allocation map export.
• Must make sure that all the interfaces are enabled.
• Fixed Ganesha not using right interface in RHEL6.5.
• Fix GPFS_FCNTL_GET_DATABLKDISKIDX fcntl API to return location info of pre allocated block correctly.
• clear sector 0 and last 4k bytes of the disk before it is created as NSD to prevent accidental GPT table recovery by uEFI driver.
• Fix a race condition problem in fileset snapshot restore tool when it tries to restore extended attributes for a directory.
• When GPFS kernel module is loaded on Linux, look up dependent symbols on demand.
• Fix stale mount handling in case of home mount issues.
• Fixed problem in scanForFileset when sgmgr resigns while the scan is in progress.
• Fixed problem where the GPFS daemon may terminate abnormally while performing encryption key rewrap (mmapplypolicy command with "CHANGE ENCRYPTION KEYS" rule). Fixed problem where mmrestorefs -j on an encrypted file system may resultin the file system being unmounted.
• Prevent multiple eviction processes from running simultaneously.
• Assert in Deque after gracePeriodThread runs.
• Update mmchmgr to pick the best candidate as new filesystem manager when user did not specify a new manager node.
• Fix a memory leak in the GPFS daemon associated with Events Exporter, mmpmon,and SNMP support.
• In GPFS systems employing GPFS Native RAID, there was a situation in which failover and failback, and disk replacement operations could hang, requiring GPFS to be restarted on the node to clear the hang. Fix is extremely low risk and highly recommended.
• Fix mmsetquota bug that returns invalid argument if a stanza contain fileset attribute along with type=FILESET.
• Fix deadlock if fs panics during E_IO err processing.
• mmdeldisk is blocked while phase3 recovery is doing deferred deletions. It is enough to wait until log recovery is done.
• Ensure SQL migration is done on GSS nodes only.
• Use maxLogReplicas instead of defaultMaxMetadataReplicas to calcuate the number of new log items when a new stripped log is added.
• Limit PORTMAP inactive failure due to DNS busy.
• Ensure that vdisks are automatically scrubbed periodically.
• Initialize the fromFsP to NULL in openArch() to guard against ever calling gpfs_free_fssnaphandle() with a bad argument. Add an informative message to look for the an error log in /tmp when the file writer pipeline is broken.
• Correct the multi-release table to avoid releasing fcntl tokens prematurely.
• Fixed race condition between two threads trying to become metanode at the same time.
• Ensured not to create file if it already exists for NFS when Ganesha is running.
• Fixed a typo in in removeOpenFileFrombgdList function that caused sig 11.
• Fix code to prevent potential GPFS daemon assert during log recovery. This problem only occurs when filesystem is 4.1 format with 4K alignment enabled (4K inode size, etc). Data replication has to be enabled and direct IO been used for write with buffer size that is not 4K aligned.
• This update addresses the following APARs: IV61626 IV61628 IV61630 IV61655 IV61988 IV61991 IV61995 IV62043 IV62091 IV62215 IV62243 IV62418.

Problems fixed in GPFS 4.1.0.1 [June 06, 2014]

• Fix thread-safe problem in dumping GPFS daemon threads backtrace.
• Fixes a problem with fsck repair of deleted root directory inode of independent filesets.
• Fixed a problem in clusters configured for secure communications (cipherListconfiguration variable containing a cipher other than AUTHONLY) which may cause communications between nodes to become blocked.
• After a file system is panicked, new lock range request will notbe accepted.
• This fix only affects customers running GNR/GSS on Linux, and who have in the past misconfigured their GNR servers by turning the config parameter "numaMemoryInterleave" off, and who experienced IO errors on Vdisks as a result of that misconfiguration. These IO errors can potentially corrupt in-memory metadata of the GNR/GSS server, which can lead to data loss later on. This fix provides a tool that can be used to locate and repair such corruption.
• Remove mmchconfig -N restrictions for aioWorkerThreads and enableLinuxReplicatedAio.
• Fixed problem when reading a clonde child from a snapshot
• Fixed a rare race condition causing the assert when two threads are attempting to do a metanode operation at the same time whilethe node is in the process of becoming a metanode.
• Fixed a deadlock in a complicated scenario involving restripe,token revoke and exceeding file cache limit.
• Fixed race between log recovery and mnodeResign thread
• E_VALIDATE errors in the aclFile after node failure
• Deal with stress condition where mmfsd was running out of threads
• Fix a problem in log recovery that would cause it to fail when replaying a directory insert record. The error only occurs for filesystems in version 4.1 format, where the hash value of the file name being inserted is the same as an existing file in the directory. The problem is also dependent on the length of the file name, and only happens if the system crashes after the log record is committed, but before the directory contents are flushed.
• Fixed the problem that was caused by a hole in the cleanBufferafter the file system panicked.
• Close a hole that fileset snapshot restore tool (mmrestorefs -j) may cannot restore changed data for a clone child file.
• Fix a rare assert which happens under low disk space situation
• Fixed deadlock during mmap pagein
• Fixed the problem of excessive RPCs to get indirect blocks and the problemof metanode lock starvation involving a huge sized sparse file.
• A problem has been fixed where the GPFS daemon terminates abnormallywhen direct I/O and vector I/O (readv/writev) is used on encrypted files,and the data is replicated, or the data must go through an NSD server.
• Fix a potential deadlock when selinux is enabled and FS is dmapi managed.
• Close a hole that fileset snapshot restore tool (mmrestorefs -j) may cannot restore a snapshot in a race condition that one restore thread is deleting a file but another restore thread is also trying to get file attributes for this file.
• Fixed a kernel oops that caused by a race in multiple NFS readson the same large file.
• mmchfirmware command will avoid accessing non-existent disk path.
• Fix a directory generation mismatch problem in an encrypted secvm file system.
• shutdown hangs in the kernel trying to acquire revokeLock
• Apply at your convenience. Even if you hit this bug, an equivalent cleanup is completed later in the command execution.
• improved stability of daemon-to-daemon communications when cipherList is set to a real cipher (i.e. not AUTHONLY).
• The serial number of physical disks is now recorded in the GNR event log, and displayed in the mmlspdisk command.
• GNR on AIX allow only 32K segment.
• Fixes a problem with fsck repair of corrupt root directory inode
• mmbackup tricked by false TSM success messages Mmbackup can be fooled by TSM output when dsmc decides to roll-back a transaction of multiple files being backed up. When the TSM server runs out of data storage space, the current transaction which may hold many files will be rolled back and re-tried with each file separately. The failure of a file to be backed up in this case was not detected because the earlier message from dsmc contained "Normal File --> [Sent]" though it was later rolled back. Fixes in tsbuhelper now detect the failure signature "** Unsuccessfull **" string and instead of simply ignoring these now will revert the changes in the shadow DB for the matching record(s). Hash table keeps track of last change in each record already, so reverting is now a legal state transition for hashed records. Reorganized some debug messages and streamlined some common code to work better. Now find 'failed' string to issue reversion updates as well. Fixed pattern matching in tsbackup33.pl to properly display all "ANS####" messages.
• Fix RO cache i/o error if mounting fs in ro mode.
• Don't release mutex if daemon death.
• Fix the path buffer length calculation to return the correct length for dm_handle_to_path() functions.
• Fix bug in mmauth that may cause duplicate configure entries and node numbermismatch in configure file.
• Fix a problem with creating directory if the parent directory has default POSIX ACL.
• mmbackup fails to read hex env values mmbackup debug values, progress reporting, and possibly other user settings may be presented in decimal or hex, especially the bit-mapped progress and debugging settings. Perl doesn't always interpret the hex values correctly unless converted with the oct() function.
• Correct an NLS-related problem with mmchdisk and similar commands.
• This update addresses the following APARs: IV60187 IV60468 IV60469 IV60471 IV60475 IV60478 IV60543 IV60863 IV60864.