Unless specifically noted otherwise, this history of problems fixed for IBM Spectrum Scale 4.1.x applies for all supported platforms.

Problems fixed in IBM Spectrum Scale 4.1.1.7 [May 11, 2016]

Fix fsck duplicate fragment problem report to be in a neater tabular format.
Fix a rare fsck deadlock that can occur during fsck termination.
Fix a bug that prevented offline fsck from reporting replica mismatches.
Enable fsck to detect and repair duplicate sub-directory entries in a directory.
Fix a problem with restripefs -R which was incorrectly setting the currentDataReplicas of logfiles.
Fix a deadlock that can occur when FPO is enabled and a node's local stripe is panicked.
Fix possible deadlock which occurs when a node loses quorum (cluster membership) because of a network adapter or network outage.
Fix fsck handling of a compressed disk address.
Fix a problem in which online replica compare reports a mismatches on the last block of an inode allocation map and a block allocation map file.
Fix a hang that can happen when unmounting the filesystem.
Fix a GPFS daemon abort that can occur when a GNR backup is performed and the server is down.
Prevent GPFS daemon from asserting on Windows when collecting debug data on waiters.
Fix a problem in which disable cluster CCR left an authorized_ccr_keys file behind which may cause a startup problem if cipherLists and or nistCompliance are changed.
Fix an assert then can occur during online replica compare when the filesystem has different data / metadata buffer sizes.
Fix a bug where offline fsck was not repairing some inode problems detected during dir scan phase.
Fix a deadlock that can occur when NFS server/remote mount did not respond after doing AFM internal mount.
Offline fsck in read-only mode will now warn about unavailable disks before scanning the file system.
Allow mmchconfig to delete an empty nodeclass from the GPFS configuration node.
Fix a daemon assert that can occur while doing prefetch reads along with readdir and lookup commands when the application nodes and the gateway nodes are the same.
Fix a node crash that can occur during the recovery of another failed node if an EventsExporter "get nodes" request is issued at the same time.
Fix an E_NOATTR link failure that can occur on a SW fileset while writing to a file and droppending and resync are being run.
Fix autoload issues where GPFS may not come up on configure servers in a SERVER based cluster if files in /var/mmfs/gen/nodeFiles are missing.
Fix a problem in which tsgescsiinfo reports invalid ESM information. This fix is required for all platforms. The condition seems to be SAS fabric related.
Make a stuck tslsenclslot easier to diagnose
Change mmbackup behavior when policy scan fails. Permit operation in a reduced-capacity to do backup and not expire when directory scan results are incomplete. When this happens, no expirations should be processed, just backup. Shadow DB lines for removed files should be left alone.
Fix an issue where CES clients fail to connect after failover on Juniper switches.
Fix unexpected empty CES IP configuration file.
Lift restrictions on -B, --max-backup-count, and --max-expire-count
The change ensures that the online replica compare does not throw false positive replica mismatches on files with last block being a sub-block.
Install at your convenience, but especially if you have been adventurous trying unsupported QOS features on a 4.1.1 system. To use QOS feature, upgrade to 4.2 or higher. This disables QOS startup.
Fix a kernel panic that can occur under a heavy write work load and a dying mmkproc thread.
Fix a problem in which an amber disk "fault" light may remain on after temporary disk unavailability. The problem is specific to the 60 disk NetApp disk enclosure only.
Fix memory tracking issue in AFM environment where gateway node memory usage appears like growing without any real memory leak. This causes replication to stop.
Fixed an issue in AFM environment where random writes to same file causes memory leak after replication.
Fix an issue in AFM environment where incorrect dependency causes resync to fail.
If asynchronous NFS/NLM locking is used this fix will prevent potential kernel crash.
Modification of ACLs via mmputacl or equivalent can render the ACL as missing on a GPFS Windows node.
This update addresses the following APARs: IV81870 IV81877 IV83264 IV83271 IV84206 IV84251 IV84252 IV84253 IV84254 IV84255 IV84270 IV84428 IV84573 IV84574 IV84576.

Problems fixed in IBM Spectrum Scale 4.1.1.6 [March 31, 2016]

Fix for an assert caused by an NSD being deleted and then quickly recreated
Fix a cluster not being able to start when a node hosting LROC disks is not available.
Fix a problem in which fsck incorrectly reports not enough memory available.
Fix a problem in which fsck patchfile apply fails when it encounters a corrupted inode.
Fix asserts on didEmpty and Signal 11 faults in delSnapshotEmpty that can occur during snapshot deletion.
Fix AFM errors that can occur when writing to a large file during failover/resync.
Fix a mmrestorefs assert which can occur during the delete clone file phase. The clone was left in a bad state during a force unlink of a fileset.
Fix ENOENT failures that can occur during a snapshot restore and during iopen64 API calls.
Fix a problem which may result in a daemon assert when running the mmcheckquota command and a snapshot is corrupted
Fix an assert exp(ibdP->llfileP == this) that can occur during an offline fsck.
Fix a daemon assert: (poolId != ((SGPoolId) -1)) in line 683 of FSTypes.h. The daemon assert could occur during mmrestripefile or mmchattr with -I yes after storage pool get deleted as part of running mmdeldisk with -p or -c option.
Fix fsck repair of inode fullblocks field.
Fix fsck handling of corrupt inode filesetId.
Fix a deadlock that can occur during a failover while a HSM application is running.
Fix assert exp(synched.isNULL()) that can occur during a high work load on a LROC disk.
Fix a problem in which the GPFS/gskit installation process or various mm* administration commands can fail if Windows OS environment variables are changed in such a way that they do not exactly match the Windows installation directory name.
Fix a problem that the primary RG server can't take back the RG after restoring pdisk paths, e.g. after cable pull, etc.
Fix a problem in AFM environment where prefetch overwrites dirty files in local updates mode.
Fix a MD5sum mismatch in data after resync operation which can occur if a resync, a touch, and a write all happens at the same time.* Fix a problem in which fsck wrongly reports holes in an ACL file.
Fix a problem in AFM environment where large ACLs cannot be replicated because of buffer allocation issue.
Fix a problem where gpfs_getacl returns a bad ACL entry when called with the GPFS_GETACL_STRUCT flag and acl_level GPFS_ACL_LEVEL_V4FLAGS.
Fix a E_ROFS write error that can occur when you write over a clone file and make it a clone parent and then run recovery.
To prevent confusion in messages between GNR, GSS, ESS products, and the GPFS file system metadata, the word "metadata" was removed from all GNR errors and log messages.
Fix a deadlock in AFM environment where peer snapshot creation could deadlock with synchronous messages like (Lookup, Open, Read etc..). This can only occur if peer snapshots are enabled.
Allow snapshots to be created while snapshots are being deleted.
Fix a problem in AFM environment where replication would stop because of error while replaying Rename operation. AFM queue will be stuck state while replaying Rename operation and no new operations will be replicated.
Fix an unexpected CES IP assignment and movement of CES nodes which are not ready to host CES IPs when the address distribution policy node-affinity is selected.
Fix the deadlock in AFM environment where readdir results in deadlock under heavy stress over GPFS backend.
Fix an gpfs.snap hang on an AFM node with stale NFS mounts.
Fix a mmapplypolicy command fail when multiple commands are issued nearly simultaneously AND tscCmdPortRange has been configured in a SONAS environment.
Fix a problem which stops autorecovery from being triggered if a node which has only dataAndMetadata disks is down.
Fix a problem in which a Windows client lost view of ACLs in mixed Linux cluster.
This update addresses the following APARs: IV78971 IV81342 IV81344 IV81347 IV81686 IV81873 IV81879 IV82179 IV82181 IV82182 IV82184 IV82238 IV82610 IV82637 IV83046 IV83110.

Problems fixed in IBM Spectrum Scale 4.1.1.5 [February 16, 2016]

Fix an issue that could cause the GPFS daemon to abnormally terminate or that could cause the reporting of incorrect performance data when GPFS SNMP subagent, mmpmon, or zimon are utilized.
Fix the code to build remote attributes during recovery when there is a version mismatch.
Fix the SFSLink failure that can occur when files are created during failbacks.
The performance of the daemon has been improved in the cases where the cipherList is set to a value other than 'empty' or AUTHONLY.
Fix a quiesce assert that can occur when files are being recovered.
Fix a problem that can occur when accessing files with managed regions.
Fix a problem that can occur during clone operations.
Fix a problem that file sizes were being set incorrectly on sparse files during failback.
Fix a problem that can occur on a live file system that results in a deleted file still existing or a created file not existing after a snapshot restore.
Fix a crash in msgMgrThreadBody that can occur during unmounting and unlinking filesets on a very busy system.
Fix a problem where mmchfs -z, -Q or --perfileset-quota may fail when multiple mmchfs commands are being performed at the same time.
Fix a problem in which a incorrect vdisk state is displayed by the mmlsrecoverygroup command during a DA rebuild.
Fix a problem in which a signal 11 in verbsDisconnect_i is seen on one node when gpfs shut down on a different node. This problem can occur if the nodes are RDMA connected and are configured with a large fabnum value.
Fix a problem with GPFS logging code that could cause GPFS daemon to die with signal 11. This problem can only occur on nodes with LROC enabled.
Fix a problem that could cause a FSSTRUCT error to be logged when reading EA from a disk. This could only occur when LROC is enabled and the EA does not fit in the inode.
Fix a daemon assert that can occur during recovery.
If a GNR system using GSS hardware uses Lenovo-branded disks, this change enables recognizing disk FRU (field replaceable unit) numbers. This simplifies service procedures, and allows disk replacement without error messages.
Fix an assert that can occur during a fileset delete. perfileset-quota needs to be enabled and the fileset needs to have quota entries.
Fix an assert that can occur after a small write of data in the middle of a clone child on a system that heavily uses clones.
Fix a hang that can occur while resync is running on a SW fileset that reaches it's hard memory limit.
Fix a mmfsd daemon crash that can occur when Zimon is used to monitor the node, and a file system is force unmounted due to some unrecoverable file system error.
Allow changing the daemon interface of a non-quorum node in a CCR enabled cluster.
Fix a daemon assert that can occur during the stopping of a NFS server and there exists a fileset with expiration enabled.
Restrict the mmchcluster command from disabling CCR in a cluster that has a CES node. Administrator must remove all CES nodes from the cluster or use the --force option to disable CCR.
Fix an incorrect fileset name being displayed by the mmlsfileset command. This can occur after deleting a dependent fileset when a snapshot exits with the fileset before it was deleted.
Fix a problem in which orphans can not be deleted from the ptrash directory.
Fix a GNR server node crash that can occur during a network failure trying to connect the GNR server pair.
Fix an assert that can occur when adding pdisks with the --replace option to a cluster and one of the pdisks is in a bad state.
Fix an assert that can occur during a snapshot restore of a sparse file with a file size close to the maximum file size limit.
Fix an assert that can occur during a fsck recreate of an ACL file.
Fix a mmbackup command failure that can occur on an AIX node when the command line arguments are too long.
Fix a problem in which a fileset is stuck in an unmounted state that can occur if the remote becomes stale and both the application node and the gateway node are the same.
Fix an assert that can occur during a multi-node fsck on a 16MB block size file system that has more then 16M inodes.
Fix a node crash that can occur during a rolling upgrade.
Fix a mmfsd node crash that can occur when NSDRaid is not enabled.
If a system built on GNR/GSS/ESS servers has been getting IO errors on GPFS file systems (reported all the way to the end user application, not internal disk IO errors on individual physical disks), and those IO errors happened exactly at a time when some pdisks were unreachable (for example due to cabling or connectivity issues), and those pdisks would have been reachable from the backup node of the GNR server, then this fix will prevent the IO errors, by failing the recovery group containing the affected vdisk over to the backup node.
Fix a problem that can occur in the mmbackup command when /tmp is full.
Fix a problem in which mmaddnode fails to copy the committed key file to the new node. This only occurs on a CCR disabled cluster and if there are 2 key files.
Fix quorum loss when the network is broken between two nodes and the cluster is configured with tiebreaker disk.
Fix a problem in which the hard memory limit is not honored when the fileset is in a disconnected state.
Fix command failures in a CCR enabled cluster on nodes that also have non-GPFSgskit packages installed.
This update addresses the following APARs: IV79340 IV79341 IV79751 IV79756 IV79761 IV79767 IV80404 IV80405 IV80407 IV80789 IV81068 IV81071.

Problems fixed in IBM Spectrum Scale 4.1.1.4 [December 15, 2015]

Fix a problem counting the number of mmpmon clients; prevent improper double close of a file descriptor.
Fix GNR AU log long waiters seen in SSD replacement.
Fix a deadlock when GPFS writes to memory mapped buffer and the same thread a lock already on it.
Fix the truncate(2) up failure issue on clone child file.
Add gpdQuorumLossShutdown to be one of the assert condition.
Fix the AFM write to sparse file to home hang issue.
Fix an issue in log code which can cause log recovery to be incorrectly skipped after a node failure. This could only occur on a 4K aligned filesystem where GPFS runs into problem completing log wrap operation.
Fix the restore failure when restoring clone children files.
Fix data mismatch on clone child file after restore.
Fix data mismatch on regular file which is not clone kind of file after restore.
Fix log writebehind code to prevent writing log record to old disk address while log file is being migrated. This issue will show up as a log recovery error if a node fails shortly after a log record was written to a wrong location.
Update log recovery code to set junction bit when replay log to recover directory for a newly created fileset. The missing junction bit can only be detected via offline fsck.
Fix the failover/resync to support outband trucking.
Fix the data inconsistency issue between cache and home during resync on appended files.
Fix the restore failure that happened at attributes restoring phase.
Fix deadlock scenario that can occur when deleting a snapshot.
Fix the ACL/EA mismatch during resync by considering ctime changed option.
With NFS backend, ATTR_MTIME_SET implies ATTR_MTIME, but GPFS ignores setattr(ATTR_MTIME_SET) if ATTR_MTIME is also not set.
Fix code to avoid high CPU usage by the mmfsd process under Windows.
Update locking code to prevent a GPFS daemon assert. The assert could happen when more than MaxFcntlRangesPerFile (default 200) advisory locks were placed on a single file.
Customer may experience signal 11 when trying to delete pdisk in the middle of RG fail over. The fix is to eliminate this problem.
Fix the dentry count leak by adding the code to call dput in error path.
Fix out of quota errors that can occur on filesystems with a format less then 1400.
Fix the mtime mismatch between cache and home for zero sized files by copying mtime from openfile to child attributes.
Apply if you use -B number with number > 2**31-1 in any of your commands or scripts.
Fix is recommended for all GNR (ESS/GSS) customers. The problem could occur in the event of an actual disk enclosure failure.
With this feature, user will be able to add a 4K native disk to existing non-4K aligned file system if the disk is used dataOnly, and the file system data block size is at least 128K, and the file system version is at least 4.1.1.4.
Fix the issue by allowing prefetch to continue if parent cannot be found for some files.
Fix the memory mapped read performance issue on AFM filesets.
Fix the mmrestorefs[479] : daemon command memory fault issue.
Fix a problem with copying key files in mmsdrrestore where the node that is being restored does not have prompt less password access to the issuing node.
Fix the case where the ESS storage enclosure slot location that is cached in the daemon can get stale and is not getting updated.
Do not allow the AioWorkerThread to steal a dirty buffer. This prevents a deadlock.
Fix the mmdiscovercomp command that is failing with "Constraint error" when trying to add servers to the component database.
Fix code to avoid quorum loss declaration of the current cluster manager, when the network is broken between two nodes.
Fix the fileset unlink hang by closing the control file before calling unmount.
If a system built on GNR/GSS/ESS servers has been getting IO errors on GPFS file systems (reported all the way to the end user application, not internal disk IO errors on individual physical disks), and those IO errors happened exactly at a time when some pdisks were unreachable (for example due to cabling or connectivity issues), and those pdisks would have been reachable from the backup node of the GNR server, then this fix will prevent the IO errors, by failing the recovery group containing the affected vdisk over to the backup node.
Add code to flush data buffers first before setting cached bit.
Fix the path to the Linux modprobe command that the mmchfirmware command uses when --type adapter is specified.
Starting with 4.1.1, GPFS changed the contents of the Linux NFS filehandle, compared to earlier versions (while still supporting older filehandles). This means if the AFM home is upgraded to 4.1.1 or later, existing AFM filesets detect a change in export since the filehandle changes and will suspend future synchronization with home. Similarly, a change from knfsd to Ganesha at home also causes a filehandle change even though the export is the same. The only solution is to resync the cache using failover which is expensive. This fix handles upgrades if home is running GPFS by detecting and upgrading cached filehandle when the filehandle changes for an inode.
Fix the mmdiscovercomp command that is failing when there are multiple building blocks.
Re-enable online replica compare and repair.
This update addresses the following APARs: IV76482 IV78653 IV78662 IV78666 IV78669 IV78672 IV78810 IV78910 IV78912 IV78913 IV78914 IV78915 IV78932 IV79336 IV79338 IV79339.

Problems fixed in IBM Spectrum Scale 4.1.1.3 [October 29, 2015]

Fix a problem in a Disaster Recovery (multi-site) environment. If a network outage prevents the two main sites from talking to each other while both sites can still communicate with the tie-breaker (single-node) site, it is possible that the cluster manager may end up moving from the primary to the backup site. That may cause the primary site to lose quorum.
Fix a PreAlloc log assert which happens when "offset + len" wraps through zero.
Fix a regression which breaks FPO locality aware restripe.
Fix the api gpfs_get_fssnaphandle_by_name to return the proper number of bytes, when called from a 32 bit application, so that the heap is not corrupted.
Fix a memory map I/O offset issue that GPFS may not handle I/O properly for very huge file.
Fix the mmrestorefs command failure on data changes restore phase.
Handle minquorumNodes correctly in CCR enabled cluster.
Fix GPFS SNMP subagent to work with newer Net-SNMP versions. This fix should be applied to any GPFS cluster node given the role of snmp_collector, if it is running RHEL 7.1, or some other Linux version that includes Net-SNMP 5.6 or beyond.
Do not return AFM-specific internal attributes in gpfs_fgetattrs().
On 2.6.39+ linux kernel, add explicit blk_start_plug/blk_finish_plug inside gpfs io submit routine, let io scheduler have more chances to merge IOs into a bigger size one.
This update addresses the following APARs: IV77541 IV77542 IV77544 IV78046.

Problems fixed in IBM Spectrum Scale 4.1.1.2 [September 10, 2015]

Avoid buffer overrun risk in AFM multi-byte scratch file name generation.
Fix the cause of a crash of the GPFS mmsnmpagentd daemon. The fix only applies to GPFS clusters where a node has been given the snmp_collector role, as seen in mmlscluster output.
Fix mmbackup which could report success even when some designated files did not back up. The count of objects backed up can become inaccurate due to a persistent problem that the reported number of objects backed up can be inflated by "dsmc" when it chooses to back up additional items such as parent directories. Correct the count of objects backed up by carefully monitoring for any possible misrepresentation from the individual dsmc commands.
Fix the mmrestorefs command failure at the attributes restore phase of the command.
mmfsadm dump improvements: add more loop restriction to exit loop after dumping all the original number of cached record addresses and improve SIGFPE support during dump.
Fix rare case of deadlock in direct IO code path when flushing the stolen buffer.
Fix memory fault (core dump), loop or hang in mmimgrestore during exit processing.
This fix affects environments installing the Object protocol on an external Keystone where the administrator wants the install to automatically create the Swift entries in the Keystone server.
When slab allocator creation fails, printk a warning message then fail mmfslinux.ko load instead of panic the kernel.
Fix a possible GPFS daemon crash when using the mmcharrier command to replace a disk in the P7 disk enclosure in which some of the disk slots were not populated. Fix is recommended for P7IH customers and not relevant to other systems.
Re-enable quota limits automatically after "mmcrfs -Q yes" and "mmchfs -Q yes". It has been disabled wrongly since GPFS v4.
Fix potential signal 11 encountered during dump of NSD IO buffers.
Fix the daemon hang during handler cleanup in AFM environment.
Fix an error when mmafmctl flushpending is invoked without fileset name.
Fix the data restore problem for the small file which only has fragment block.
This fix affects environments running the Object protocol with a locally-installed Keystone server with SSL support.
Fix assert that might occur on systems configured with a small shared segment under stress workload that includes metadata updates and frequent buffer steals.
Fix code to avoid removing wrong address during deletion of addresses from the cesiplist configuration file.
Increased stability of the library used to retrieve keys used for file encryption from ISKLM.
RecLockModuleReset call to __posix_lock_file encounters bad file pointer
Fix a deadlock caused by not releasing the DMAPI lock in failure path of AFM read.
Fix a problem that suspended disks are still marked as "tobeemptied" after successful restripe.
Migrating files in RO fileset causes SetXAttr to be queued at gateway node.
Fix the undefined symbols in 32-bit version of libgpfs.so.
Fix null pointer dereferencing in AFM expiration code by limiting it to work only on valid and registered fileset handlers.
The GSKit toolkit has been updated to version 8.0.50.47, which(1) fixes the vulnerability described in CVE-2015-1788 and (2) improves the performance of secure sends (cipherList is set to a cipher other than empty or AUTHONLY)
Fix a problem that the GSS/ESS component database information can appear out of sync.
Optimize cifsProcess::isRegistered when the hash chain is empty
Fix a specific case where the remote cluster is removed before cleaning up the remote mount entries when using mmremotefs delete.
Upgraded LROC to support new NSD disk layout.
Drop the GNR track mutex when trying to acquire the log mutex
Fix signal 11 in saveInodePts when configured to use a localCache.
Fix performance degradation under a workload accessing a large number of files, due to unnecessary atime refresh messages.
Improve performance for workloads with large numbers of files on systems with fast metadata storage.
Update code to ignore EINPROGRESS error from flush when setting up pipe for invoking external script from GPFS daemon.
Fix signal 11 in daemon caused by removing a localCache device.
When mmchfs is run with a rapid repair option this fix will check to see if the file system is unmounted before executing the command. An error is issued if the file system is mounted.
Update the threshold to print 'memory usage approaching the limit' warning message that was triggered too early.
Fix a problem in the AIX operating system, where some system calls like open() may set errno to EPERM, even if returning successfully, when run from non-root users. System calls like shmat() (when used to map a file) may fail with the same value of errno.
Relax server license requirement for NSD disks in system.log pool
This update addresses the following APARs: IV75396 IV75999 IV76016 IV76017 IV76018 IV76019 IV76020 IV76383 IV76455 IV76457 IV76458 IV76461 IV76467 IV76471 IV76473 IV76475 IV76518 IV76759

Problems fixed in IBM Spectrum Scale 4.1.1.1 [July 30, 2015]

Fix a rare case that could cause mmsnmpagentd to consume up to 100% of CPU when GPFS daemon terminates. Only affects clusters where a node is given the SNMP collector role.
Avoid rare kernel assert while deleting many snapshots concurrently on a sluggish system.
The command mmrpldisk now reports no space error instead of panic the GPFS file systems with several almost full disks.
Provide a default user exit for nodeLeave event for FPO clusters so that the disks could be marked as down and the data integrity is not compromised.
Print accurate remaining redundancy in the log when rebuild fails due to insufficient disk space.
Fix "No disk name found" error when all of the disks are either in "emptied" or "to be emptied" state.
nsdperf can hang when used on large number of linux nodes
Change gpfs_prealloc not to preallocate blocks when the requested preallocate size is within the last block of the file but less than the file size. The allocation blocks are rounded to GPFS block boundaries when the file has fragments.
mmdeldisk (relocation of aclFile blocks) results in lost ACL
Fix a rare kernel crash case in incompleteAioListRemove when doing AIO on Linux.
Fix a deadlock resulted from running fsck and recovery in parallel.
Fsck reports false positive DA corruption
Fix a problem that makes file blocks not distributed in metablock unit among nodes when the FPO file system has not enough failure groups.
Enhance FPO autorecovery log for clarity
Fix a problem encountered when dumping buffers with NSD checksum errors.
When a vdisk I/O times out, failover the recovery group to the back up node.
Prevent an assert due to a race condition while both creating and deleting snapshots concurrently.
By moving the truncation operation of clone child files to the later delta restoring phase, the failure of truncation on clone child files is avoided.
By extracting the right log file name from the input of "device" of mmrestorefs command, user should not see this internal failure error when the restore process failed.
Fix a deadlock during Ganesha queue clean up. Now when the daemon crash we don't clean the Ganesha queue using the Ganesha thread, clean it later during SG cleanup.
Fix slow performance of some administration commands when CCR (Cluster Configuration Repository) is enabled
Correct a small vulnerability in takeover after SG manager failure during a snapshot command.
Fix secondary kernel exception (get_stcP) on Linux cNFS server
Enhance mmfsctl to work with topology vector failure group, NSD stanza file.
Fix performance issues in ESS/GSS clusters in very high stress. This fix applies to customer with client nodes in a ESS/GSS cluster containing Connect-IB adapter.
Fix memory fault (core dump) in mmimgrestore during exit processing
Improve the performance of communication across daemons when the 'cipherList' configuration parameter is set to something other than empty or AUTHONLY.
Ganesha: file descriptor was used after it was released causing assertion. Now the release is done at exit after all references the the files are done.
Provide inode number information to an assertion within the low-level file write operation.
Fix assert "openInstCount >= 0" under stress workload that includes file deletions.
kxSendFlock needs to copyin user objects
Fix a problem that the disk failed LED may not lit when setting the disk state to failed
Fix a problem with AIO write pass the end file where file size change may be lost if GPFS daemon fails or file system panics shortly after write was completed.
Fix a problem that mmdf show 0 free blocks for suspended disks
Fix a kernel panic due to NULL pointer dereference during hard reboot of the partner node in Ganesha environment
Fix a problem with DIRECT_IO write which can cause data loss when file system panic or nod e fails after write pass end of file using DIRECT_IO causes an increase in file size. The file sizeincrease could be lost.
Enhanced the file system inconsistency state check during restore process and then graceful exit if detected.
Fix the problem that data missed to write to new allocated datablock when file was expanded to size larger than old allocated datablock.
Fixed the problem with VMWare NFS v3 client in Ganesha environment by providing an option to enable short_file_handle that VMware NFS client is using.
Fixed a replicas mismatch problem that was caused by using wrongblock index in the indirect block.
gpfs hadoop connector supports Hadoop 2.7.x release
gpfs hadoop connector supports hdfs:// schema
This update addresses the following APARs: IV74661 IV74686 IV74697 IV74732 IV75108 IV75394

Summary of changes for IBM Spectrum Scale version 4 release 1.1 as updated, June 2015

Changes to this release of the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library include the following:

Active file management asynchronous fileset-level data replication for disaster recovery (DR)
Asynchronous replication of data at the file level enables you to create a primary(active)/secondary(passive) relationship at the fileset level. Data is asynchronously replicated to the secondary on a periodic basis. To enable this function, ensure that you run the following commands: * If you are migrating from a previous release, run the mmchconfig release=LATEST command. * Run the mmchfs -V full command.
Cluster Configuration Repository (CCR) Enhancements were made to restore broken configuration and files to bring a cluster back online or a broken node to a working state. In the case of a disaster recovery setup, steps are provided to downgrade the quorum assignments when half or more of the quorum nodes are no longer available at one of the sites. Consult the "Establishing disaster recovery for your GPFS cluster" topic in the IBM Spectrum Scale: Advanced Administration Guide.
Cygwin 64-bit version requirement for Windows nodes The 32-bit version of Cygwin is no longer supported for Windows nodes running GPFS. Users that are running GPFS 4.1 with the 32-bit version of Cygwin installed must upgrade to the 64-bit version of Cygwin before installing IBM Spectrum Scale 4.1.1. Users with SUA on GPFS releases prior to 4.1 should upgrade directly to the 64-bit version of Cygwin.
Data collection for expelled nodes When a node is about to be expelled for unknown reasons, debug data is collected automatically to help find the root cause.
Deadlock amelioration Deadlock breakup requests can be issued on demand at a time that is chosen by a system administrator. A user callback for the deadlockOverload event can be added to notify a system administrator to check the system and workload for an overload condition.
File Placement Optimizer (FPO) FPO enhancements deliver the ability to change block allocation of an existing file with the mmrestripefile and mmchattr commands and efficient removal of disks when disks have already been emptied with the auto recovery process. Auto recovery has been optimized to handle multiple failure and recovery events more efficiently.
Fileset-level integrated archive manager (IAM) modes Fileset-level integrated archive manager (IAM) modes give users the ability to set four different IAM modes at the fileset level, including the root fileset, so that users can modify the file-operation restrictions that normally apply to immutable files. For more information, see the following: * topic about immutability and appendOnly restrictions in the Information Lifecycle Management chapter of the IBM Spectrum Scale: Advanced Administration Guide * mmchfileset and mmlsfileset command descriptions in the IBM Spectrum Scale: Administration and Programming Reference To enable this function, ensure that you run the following commands: * If you are migrating from a previous release, run the mmchconfig release=LATEST command. * Run the mmchfs -V full command.
GPFS Native RAID (GNR) and Elastic Storage Server (ESS) documentation The documentation for GNR and ESS was removed from the information units in the IBM Spectrum Scale library. This includes GNR commands, GNR callbacks available to the mmaddcallback command, vdisk performance monitoring with the mmpmon command, messages in the ranges 6027-1850 - 6027-1899 and 6027-3000 - 6027-3099, and the chapter in the IBM Spectrum Scale: Advanced Administration Guide titled GPFS Native RAID (GNR). For more information about GNR, see GPFS Native RAID: Administration. For more information about ESS, see Deploying the Elastic Storage Server.
Hadoop support Hadoop support was expanded from FPO storage to shared storage. This allows data stored in current GPFS clusters using shared storage to be accessible to Hadoop applications. IBM Spectrum Scale Hadoop Connector has been enhanced to transparently support both FPO based storage pools to leverage data locality and shared storage where locality information is not applicable. This allows FPO and shared storage pool to be used within the same file system, which allows Hadoop applications to access data in the entire file systems transparently. IBM Spectrum Scale Hadoop Connector fully supports Hadoop version 2.5, and it can also be used with Hadoop version 2.6 in compatibility mode (Hadoop file system APIs in 2.6 are not yet implemented). The mmhadoopctl command was introduced to simplify IBM Spectrum Scale Hadoop Connector configuration and management.
Inode expansion optimization In this release, inode expansion, which allows dynamic growth of inodes, is optimized to reduce the contention that can flare up during bursts of file creates. To enable this function, ensure that you run the following commands: * If you are migrating from a previous release, run the mmchconfig release=LATEST command. * Run the mmchfs -V full command to enable all of the new functionality that requires different on-disk data structures. For more information, see the topics on completing migration and use of disk storage and file structure in file systems in the IBM Spectrum Scale: Concepts, Planning, and Installation Guide.
Installation toolkit The installation toolkit can be used to do the following: * Install and configure GPFS. * Add GPFS nodes to an existing cluster. * Deploy and configure SMB, NFS, OpenStack Swift, and performance monitoring tools on top of GPFS. * Configure authentication services for protocols. * Upgrade GPFS and protocols. For details, see the spectrumscale command description in the IBM Spectrum Scale: Administration and Programming Reference.
Multi protocol data access Data access to a shared storage infrastructure through enhanced protocol support for NFS, SMB, and Swift Object. For more information, see the IBM Spectrum Scale: Advanced Administration Guide and the IBM Spectrum Scale: Administration and Programming Reference.
Performance improvements for mmfsck The mmfsck command can now store information that is found during a scan of the file system into a patch file. The information in the patch file can then be used as input to repairing the file system. Using a patch file to repair the file system prevents an additional scan before starting the repair actions. For more information, see the mmfsck command description in the IBM Spectrum Scale: Administration and Programming Reference.
PIT inode list The parallel inode traversal (PIT) scan used for the mmchdisk, mmdeldisk, mmrestripefs, and mmrpldisk commands has now been updated to produce a list of inodes with interesting attributes, for example: those having broken disk addresses or those being ill placed. While the mmfileid command can be used to list files with broken disk addresses, this can be a slow process. Two new optional parameters, --inode-criteria CriteriaFile and -o InodeResultFile have been added to the more commonly-used mmchdisk, mmdeldisk, mmrestripefs, and mmrpldisk commands. These parameters allow you to find files matching certain criteria without a separate invocation of mmfileid. With this new feature, you can easily find the interesting files and their inode numbers. The output file will contain a list of inode numbers that meet the specified flags along with the name of the flag and the file type. For more information about these commands and for a description of the optional parameters and flags, see the commands in the IBM Spectrum Scale: Administration and Programming Reference. To enable this function, ensure that you run the following commands: * If you are migrating from a previous release, run the mmchconfig release=LATEST command. * Run the mmchfs -V full command to enable all of the new functionality that requires different on-disk data structures.
Policy improvements: This release includes the following policy improvements: mmapplypolicy --sort-command SortCommand The mmapplypolicy --sort-command parameter allows you to specify an alternative sort command to be used, rather than the default sort command provided with the operating system. Implicit SET POOL 'first-data-pool' rule For file systems that are at or have been upgraded to 4.1.1, the system recognizes that, even if no policy rules have been installed to a file system by mmchpolicy, data files should be stored in a non-system pool if available (rather than in the system pool, which is the default for earlier releases). For more information, see the following: * Information Lifecycle Management chapter in the IBM Spectrum Scale: Advanced Administration Guide * mmchpolicy command description in the IBM Spectrum Scale: Administration and Programming Reference
Quota management Quota management improvements for file system format 4.1.1 and higher include: * Allowing quota management to be enabled and disabled without unmounting the file system. To enable this function, ensure that you run the following commands: * If you are migrating from a previous release, run the mmchconfig release=LATEST command. * Run the mmchfs -V full command.
Read replica policy In a file system with replicas, there are replicas for each data block stored in different disks in different failure groups. Now, using the readReplicaPolicy attribute of the mmchconfig command you can specify the location from which the policy is to read replicas. readReplicaPolicy lets you specify that the first replica be read, the local or closest replica, or the fastest. For more information, see the mmchconfig command in the IBM Spectrum Scale: Administration and Programming Reference.
Performance Monitoring Tool The Performance Monitoring tool aims to provide performance information after collecting the metrics from GPFS and protocol nodes using the mmperfmon query command with an appropriate query. The tool helps in detecting performance issues and problems. The predefined queries and metrics help in investigating every node or any particular node that is collecting metrics. For more information, see the following: * "Performance Monitoring tool overview" topic in the IBM Spectrum Scale: Advanced Administration Guide * mmperfmon command description in the IBM Spectrum Scale: Administration and Programming Reference
Documented commands, structures, and subroutines The following lists the modifications to the documented commands, structures, and subroutines: New commands The following commands are new: * mmces * mmdumpperfdata * mmhadoopctl * mmnfs * mmobj * mmperfmon * mmprotocoltrace * mmsmb * mmuserauth * spectrumscale New structures There are no new structures. New subroutines There are no new subroutines. Changed commands The following commands were changed: * gpfs.snap * mmaddcallback * mmafmctl * mmafmlocal * mmapplypolicy * mmbackup * mmbuildgpl * mmchconfig * mmchdisk * mmcheckquota * mmchfileset * mmchnode * mmchpool * mmchpolicy * mmcrcluster * mmcrfileset * mmdeldisk * mmedquota * mmfsck * mmlscluster * mmlsfileset * mmlsfs * mmlspolicy * mmlsquota * mmpsnap * mmrepquota * mmrestorefs * mmrestripefile * mmrestripefs * mmrpldisk Changed structures There are no changed structures. Changed subroutines There are no changed subroutines. Deleted commands There are no deleted commands. Deleted structures There are no deleted structures. Deleted subroutines There are no deleted subroutines.
Messages The following lists the new, changed, and deleted messages: New messages 6027-962, 6027-2145, 6027-2230, 6027-2234, 6027-2235, 6027-2238, 6027-2240, 6027-2241, 6027-2242, 6027-2245, 6027-2246, 6027-2247, 6027-2248, 6027-2249, 6027-2250, 6027-2251, 6027-2252, 6027-2253, 6027-2254, 6027-2255, 6027-2256, 6027-2257, 6027-2258, 6027-2259, 6027-2260, 6027-2261, 6027-2262, 6027-2263, 6027-2264, 6027-2265, 6027-2266, 6027-2267, 6027-2268, 6027-2269, 6027-2270, 6027-2271, 6027-2272, 6027-2273, 6027-2274, 6027-2281, 6027-2282, 6027-2283, 6027-2284, 6027-2285, 6027-2286, 6027-2287, 6027-2288, 6027-2289, 6027-2290, 6027-2291, 6027-2292, 6027-2293, 6027-2294, 6027-2295, 6027-2296, 6027-2297, 6027-2298, 6027-2299, 6027-2300, 6027-2301, 6027-2302, 6027-2303, 6027-2304, 6027-2305, 6027-2306, 6027-2307, 6027-2308, 6027-2309, 6027-2310, 6027-2311, 6027-2312, 6027-2313, 6027-2314, 6027-2315, 6027-2316, 6027-2317, 6027-2318, 6027-2319, 6027-2320, 6027-2321, 6027-2322, 6027-2323, 6027-2324, 6027-2325, 6027-2326, 6027-2327, 6027-2329, 6027-2330, 6027-2331, 6027-2332, 6027-2333, 6027-2334, 6027-2335, 6027-2336, 6027-2337, 6027-2338, 6027-2339, 6027-2340, 6027-2341, 6027-2342, 6027-2343, 6027-2344, 6027-2345, 6027-2346, 6027-2347, 6027-2348, 6027-2349, 6027-2350, 6027-2351, 6027-2352, 6027-3255, 6027-3256, 6027-3257, 6027-3306, 6027-3307, 6027-3308, 6027-3309, 6027-3310, 6027-3311, 6027-3312, 6027-3313, 6027-3314, 6027-3315, 6027-3316, 6027-3551, 6027-3552, 6027-3553, 6027-3554, 6027-3579, 6027-3580, 6027-3581, 6027-3708, 6027-3709, 6027-3710, 6027-3711, 6027-3712, 6027-3713, 6027-3714, 6027-3715, 6027-3716, 6027-3717, 6027-3718, 6027-3719, 6027-3900, 6027-3901, 6027-3902, 6027-3903, 6027-3904, 6027-3905, 6027-3906, 6027-3907, 6027-3908, 6027-3909, 6027-3910, 6027-3911, 6027-3912, 6027-4000, 6027-4001, 6027-4002, 6027-4003, 6027-4004, 6027-4005, 6027-4006, 6027-4007, 6027-4008, 6027-4009, 6027-4010, 6027-4011, 6027-4012, 6027-4013, 6027-4014, 6027-4015 Changed messages 6027-625, 6027-872, 6027-1305, 6027-2181, 6027-2183, 6027-2229, 6027-2714, 6027-2715, 6027-2758, 6027-3248, 6027-3249 Deleted messages 6027-2622, 6027-2632, 6027-3511, 6027-3514, 6027-3515, 6027-3516, 6027-3536, 6027-3544

Problems fixed in GPFS 4.1.0.8 [May 26, 2015]

Correct a small vulnerability in takeover after file system manager failure during a snapshot command.
The code change ensures that online replica compare tool does not report false positive mismatches when the file system has suspended disks.
Fix an AFM recovery issue during the fileset unlink.
Fix a problem when determining whether copy-on-write is needed or not in the presence of snapshots. Sometimes this problem may result in spurious write operation failures (especially, but not limited to file/directory creation).
Fix a hang in mmrestripefs, which may also result in waiters for "PIT_Start_MultiJob". The problem may happen if the set of nodes specified in the '-N' option to the command includes nodes which are still in the process of being started (or restarted).
mmcrsnapshot, mmdelsnapshot and mmfileset commands quiesce the file system before they start actual work. During that quiesce if a thread doing file deletion of an HSM migrated file is stuck waiting for recall, since that recall could take long time due to slow tapes for example, then the mm commands could time out. This fix allows those commands to proceed while a deletion is waiting for recall.
Close a very small window of deadlock caused by releasing the kssLock and and calling cxiWaitEventWakeupOne when a thread not waiting for the exclusive lock is waken up and leaving the thread actually waiting for the lock sleeping and waiting.
Avoid a GPFS crash when running mmrestorefs or mmbackup where there are deleted filesets.
Enable offline fsck to validate extended attribute file
Fix a problem with directory lookup code that can cause FSErrInodeCorrupted error to be incorrectly issued. This could occur when lookup on '..' entry of a directory occurs at the same time as its parent is being deleted.
Ensure that EA migration to enable FastEA support for a file system does not assert for 'Data-in-Inode' case under certain conditions
Enable online fsck to fix AFM pre-destroyed inodes. Use PIT to cleanup unlinked inodes in AFM disabled fileset.
Update allocation code to close a small timing window that could lead to file system corruption. The problem could only occur when a GPFS client has a file system panic at the same time as the new file system manager is performing a take over after the old manager resigned.
Fix a signal 11 problem in multi-cluster environment when gpfs daemon relay the fsync request through metanode but the OpenFile got stolen on the metanode in the middle.
Remove confusing trace stop failed error messages on Windows.
The privateSubnetOverride configuration parameter may be used to allow multiple clusters on the same private subnet to communicate even when cluster names are not specified in the 'subnets' configuration parameter.
This fix indicates that mmfileid command will not work if there is only GPFS express edition installed.
Fix a workload counter used for NVRAM log tip I/O processing queues. Recommended if NVRAM log tip is in-use.
Potentially avoid crash on normal OS shutdown of CNFS node.
Fix issue where file create performance optimization was sometimes disabled unnecessarily.
In a cluster configured with node quorum, fix a problem where, if the cluster manager fails and the cluster is left with only the bare-minimum number of nodes to maintain node quorum, the cluster may still lose quorum.
Enable offline fsck to fix AFM orphan directory entries in single run
Fix a problem where the number of nodes allowed in a cluster is reset from 16384 to 8192.
This affects GSS/ESS customers who are using chdrawer to prepare to replace a failed storage enclosure drawer on an active system.
Correct a problem in the v4.1 release with directory listings in file systems created prior to v3.2.
Fix a problem that a race between log wrap and repair threads caused checksum mismatch in indirect blocks.
Fix a daemon crash in AFM ensuring that the setInFlight() method have positive 'numExecuted' value while calculating the average wait time of the messages.
Fix a problem on GPFS CCR cluster where GPFS commands may not work on inactive configuration servers after generated new security key.
Fix command poor performance on cluster that has no security key.
Fix a problem with DIRECT_IO write which can cause data loss when file system panic or node fails after a write passes the end of file using DIRECT_IO and causes an increase in file size. The file size increase could be lost.
File cache filled-up with deleted objects (Linux NFS)
Fix a hardlink creation issue by handling the E_NEEDS_COPIED error in SFSLinkFile function for AFM files.
Fix handling of policy rules like ... MIGRATE ... TO some-group-pool THRESHOLD (hi,lo) ...
The /var/mmfs/etc/RKM.conf configuration file used to configure file encryption now supports a wider set of characters.
Trigger a back-off when 90% of the configured hard memory limit is hit during queuing of AFM recovery operations.
ESS customers, using zimon, may see GPFS daemon crashes in the performance monitoring code.
ESS customers, using zimon, may see GPFS daemon crashes in the performance monitoring code.
Add support for multiple RDMA completion threads and completion queues
Fix signal 11 in verbs::verbsCheckConn_i
Fix signal 11 in runTSPcache caused by a uninitialized variable in error paths.
mmauth inadvertently change cipherList to an invalid string. Changed Externals: New messages: GPFS: 6027-3708 [E] Incorrect passphrase for backend '%s'. GPFS: 6027-3709 [E] Error encountered when parsing line %d: expected a new RKM backend stanza. GPFS: 6027-3710 [E] Error encountered when parsing line %d: invalid key '%s'. GPFS: 6027-3711 [E] Error encountered when parsing line %d: invalid key-value pair. GPFS: 6027-3712 [E] Error encountered when parsing line %d: incomplete RKM backend stanza '%s'. GPFS: 6027-3713 [E] An error was encountered when parsing line %d: duplicate key '%s'. GPFS: 6027-3714 [E] Incorrect permissions for the /var/mmfs/etc/RKM.conf configuration file. Deleted messages: GPFS: 6027-3536 [E] Incorrect passphrase '%s' for backend '%s'. GPFS: 6027-3511 [E] Error encountered when parsing '%s': expected a new RKM backend stanza. GPFS: 6027-3515 [E] Error encountered when parsing '%s': invalid key-value pair. GPFS: 6027-3514 [E] Error encountered when parsing '%s': invalid key '%s'. GPFS: 6027-3516 [E] Error encountered when parsing '%s': incomplete RKM backend stanza '%s'. GPFS: 6027-3544 [E] An error was encountered when parsing '%s': duplicate key '%s'.
This update addresses the following APARs: IV71419 IV71569 IV71601 IV71607 IV71613 IV71616 IV71628 IV71633 IV71634 IV71636 IV71648 IV71692 IV71815 IV72029 IV72033 IV72039 IV72042 IV72048 IV72684 IV72687 IV72688 IV72694 IV72695 IV72698 IV72700 IV72890.

Problems fixed in GPFS 4.1.0.7 [March 12, 2015]

Fix a problem with block allocation code, where E_NOSPC error could be incorrectly returned after running out of disk space in one failure group. This problem only affects file systems with data replication.
When a GSS logTipBackup pdisk fails, mmlsrecoverygroup output will now display offline (as opposed to error)for the affected logTipBackup vdisk.
Fix a problem in an FPO environment which may cause auto recovery to fail to suspend some down disks.
Fix a problem when deleting files from independent fileset which causes unnecessary recalls when there are no snapshots.
Ensure to check null pointer when memory allocation fails while Ganesha is active
Enforce the same declustered array (DA) name for the old pdisk and the corresponding new one when replacing a pdisk with mmadpdisk --replace
Fix problem that may cause assertion '!ofP->destroyOnLastClose' when the file system is mounted in RO mode on some nodes and in RW more on others.
Enhance the handling of TSM summary output in mmbackup.
Fix a deadlock occurring in a heavy DIO and mmap workload on Linux
Fix fsck handling of ToBeDeleted inode map bits
Fix a problem where, when migrating a block, mmrestripefile/mmrestripefs -r should but does not comply with the file's WADFG attribute if that file belongs to a FPO pool which has WAD=0, BGF=1 attribute.
Fix bug in change license in CCR cluster.
Fix a problem that might cause auto recovery failure in FPO cluster.
Fix code to calculate the correct payload for a remote procedure call during file updates to the GPFS internal configuration repository. The problem may cause assertion 'bufP - msgP == len'.
Fix a problem that might cause no space error even if there is free disk space.
Protect fcntl kernel calls against non-privileged callers.
Exclude sensitive files from gpfs.snap collection.
GPFS command hardening.
Enable dynamically switching from cipherList=EMPTY to cipherList=AUTHONLY without bringing down the entire cluster.
This update addresses the following APARs: IV68842 IV69086 IV69619 IV69657 IV69702 IV69704 IV69705 IV69706 IV69707 IV70610.

Problems fixed in GPFS 4.1.0.6 [February 5, 2015]

Customer can change the spares of an existing DA when adding new pdisk.
Update code to prevent a deadlock that could occur when multiple threads try to create the same file in a directory at same time.
When E_NOMEM occurs on a token manager during token transfer, try throw out half of all OpenFile objects, regardless of which file system they belong to. Also limit implicitly set maxStatCache(MSC) value to 10,000.
Reduce memory utilization for GPFS RDMA QPs and fix a problem with Connect-IB when verbsRdmaSend is enabled.
Fixed the assert by mapping the error to permission error as expected.
Fixed a problem where creating a clone copy file, in an encrypted file system may result in threads being blocked or in an abnormal daemon termination.
Correct tsbuhelper updateshadow command to recognize several circumstances when gencount changes even for files not yet backed up, or changed.
Fixed the problem by changing the implementation of flushpending while operations get requeued.
In FPO, use disks evenly when doing mmrestripefs -r or mmdeldisk.
Fix is especially recommended for GSS systems on the Power platform.
Fixed the problem by allowing prefetch recovery to run only if all the nodes in the cluster are all at at least minReleaselevel 1410.
Fixed the problem by clearing CA_STATE|CA_CREATE bits after local rm is done.
Fix a defect which may cause data consistency problem if one runs mmrestripefile after reducing replica level.
Fix is recommended for all GSS systems using NVRAM log tip devices.
When creating FPO file system, always use 'cluster' layoutMap if allowWriteAffinity is yes and layoutMap is unspecified.
Fixed a problem that enable file system quota would hit an assert.
"Waiting for nn user(s) of shared segment" messages on shutdown.
Fixed the problem by reading inmem attrs for dump command instead of reading it from disk.
Fsck reports false corruption in inode alloc map due to rounding on wrong sector size.
Fix code that could cause mmclone copy to fail incorrectly with EINVAL. This problem could only occur when source is file in GPFS snapshot.
Fix GNR bug related to a race condition that cause recovery failure during startup or failover.
Fix a mmdeldisk problem caused by disabled quota files placed in the system pool.
Fixes a problem where fsck hits signal 8 during inode validation
Make sure that the inode pointed to by different dentry is not the same inode to prevent possible deadlock by 2 diferent dentries pointing to the same inode and trying to lock the same inode twice.
Fix potential loss of IO error in linux io_getevents() call when enableLinuxReplicatedAio is enabled (3.5.0.14+). Fix a problem that returns 'error 217' improperly when do Direct IO on a replicated file which haspartial replicas on unavailable disks (4.1.0+).
Fix a stack corruption issue.
Fix a linux lookup crash issue.
Apply if secrecy of file metadata (pathnames, attributes and extended attributes) is a concern.
Revised disk selection algorithm to ensure no reuse can exist in a map entry being modified by rebuild or rebalance. Prior algorithm allowed this type of reuse and could lead to lessening the failure tolerance, trailer validation errors, and assert crashes in various GSS subsystems.
Ensure that GPFS uses only secure CBC padding schemes when exchanging data over TLS. This affects customers who have set cipherList to a real cipher (i.e. not AUTHONLY) in order to authenticate and encrypt data passed between nodes/clusters. This also affects customers who use file-level encryption.
Fix GPFS_FCNTL_GET_DATABLKLOC API, make it returns correct disk ID for data in inode files.
This fix is required for GSS server nodes in which multiple names assigned to the node differ only in the domain portion of the name.
Correct a problem leading to misleading tsgskkm system clock complaints.
Fix is recommended in all GNR configurations.
This update addresses the following APARs: IV67901 IV68007 IV68059 IV68060 IV68062 IV68064 IV68065 IV68096 IV68491 IV68493.

Problems fixed in GPFS 4.1.0.5 [December 5, 2014]

Fix an alloc cursor issue in block allocation code that may lead to spurious no space error in FPO file system.
Fixed code to consider flags during a disks challenge of the current cluster manager.
This fix applies to GNR/GSS customers that are adding additional server nodes and cluster is created using default block size.
Reduce number of nsdMsgRdmaPrepare messages sent.
Fix GSS bug related to concurrent overlapping read operations during failover/error scenarios.
Fixed a problem that the range size was initialized to wrong value with meta data-block size instead of the correct data block size.
Fix the problem that for certain configuration of FPO cluster, replicas failed to involve all LGs.
Redirect automatic recovery's tsrestripefs output to /var/adm/ras/restripefsOnDiskFailure.log
fix problem with verbsRdmasPer[Node Connection] set to a value of 1.
Reduce CNFS failover time on systems with large list of exports.
Allow disk addresses in inode 5 (Extended Attribute File) be be found by the mmfileid command.
Fix GSS bug related to mixed read-write workload with a working set size that matches GSS cache memory size.
Fixed a problem that range number was initialized to wrong value in a different meta data-block size environment anda full block write after lseek without placement installed.
fcntl revokes referencing a completed/freed request.
Hadoop File System API open() in connector throws exception as hdfs when user has no permission to access a file.
Fix bug where mmsetquota set the inode limit to unlimited if only changeblock quota is requested and vice versa.
If the user of a GSS system had previously changed the slow disk detection parameters manually to the following values: nsdRAIDDiskPerformanceMinLimitPct=50 and nsdRAIDDiskPerformanceShortTimeConstant=25000, then they can now remove the manual setting; but they don't have to remove it.
Fixed problem where the gpfs daemon gets sig11 when application calls GPFS_FCNTL_GET_DATABLKLOC api in mixed PPC64/X64 cluster.
This update addresses the following APARs: IV66617 IV66620 IV66621 IV67005 IV67006.

Problems fixed in GPFS 4.1.0.4 [October 31, 2014]

Fix code to prevent a GPFS daemon assert that could occur after automatic remount of filesystems. The problem only occurs on Linux node when user issued GPFS commands to access the filesystem before automatic remount has completed.
Fix "subscript out of range" error in recoverFailedDisk.
Correct --single-instance option for mmapplypolicy runs against directory.
The fix avoids a bug where pdisks become "missing" due to descriptors being overwritten. The fix is recommended if system firmware upgrades are applied.
Fix code used to check user provided buffer for NULL where it can cause writev() to incorrectly fail with EINVAL. Both readv()/writev() could be affected.
Fixed the problem that O_DSYNC was not honored on data write to small files.
Fixed Ganesha thread deadlock caused by Ganesha up-call thread's getting byte range lock that was just released.
Fix ibv_modify_qp error 22 when RDMA connecting client mlx4_0 port 2 HCA to server single port qib0 HCA.
Fix rare race condition that could result in a deadlock when moving data from the inode into a regular data block.
Too many RDMA connection created between nodes.
Fix Signal 11 on Connect-IB.
Make sure that FHs are still pointing to directories after we get the lock for rename.
1) Make sure stop Ganesha as soon as GPFS daemon cleanup starts 2) Make sure that FHs are still pointing to directories after we get the lock for rename.
Fix problem where AFM does not correctly replay Asynchronous I/O (AIO) writes, such as with aio_write(), when the file is opened with O_DIRECT flag. The problem may cause files to exhibit inconsistency between cache and home.
Fixed a timing issue which may cause AFM to miss replaying updates to home, if the update happens after a period of long inactivity. Disable a three-phase protocol during which requests are dropped while the gateway nodes are temporarily made inactive so they can be marked clean.
This update addresses the following APARs: IV65497.

Problems fixed in GPFS 4.1.0.3 [September 18, 2014]

The patch fixed mmfsd crash problem when restriping FS and adding disks running at the same time.
Fixes a problem may cause mmchdisk start to hang if there are too many pools or diskless nodes in a FPO cluster.
Fix a bug in mmdeldisk/mmadddisk that may cause file system to become unknown to GPFS if the name of that file system contains special char.
Fixed the daemon assert by allowing the async reads as dependents on lookup.
Fix local read only cache code that could caused GPFS daemon assert during file system unmount when local read only cache is in use.
Fixed a possible cause of deadlock when mmpmon or the GPFS SNMP subagent does node registration.
Improve handling and reporting of certain types of corrupted directory blocks.
Fix a defect in the fileset snapshot restore tool that it may cannot restore a file properly when the file's link count is zero but there is still open instance for the file.
mmlsfileset command used to display expiration timeouts for NON-RO fileset so if there any scripts which was looking for this will break.
Fix code that can cause GPFS daemon to assert when running mkdir on an AFM/SECVM enabled filesystem with metadata replication. The problem will only occur when there is no AIX node and GPFS can't allocate disk space for both replica.
Fix a truncate() issue. When truncate to a position which is beyond end of file on a FPO file system, the filesize could be set wrongly.
Apply if you are troubled by mmapplypolicy not terminating promptly and cleanly when ENOSPC is encountered.
Fix an assertion during mmrestoreconfig caused by quota file migration to reserved file.
Fixed a rare problem in background deletion code due to uninitialized list pointers.
Callback/user exit support is added for new event "afmFilesetUnmounted" which gets called when fileset is unmounted.
prevent the GPFS daemon from running into the assertion when a GSS vdisk client IO fails.
Fix race condition when renaming files.
Reduce the kernel stack usage in the write code path to void potentional kernal stack overflow.
This fix improves the GPFS tool useability to use mmaddcallback command to constantly monitor a desired file system status.
Avoid possible core dump by protecting pclose() call if broken pipe is known. Also, add LCL_ESTALE to list of fatal conditions.
This fix will detect the failure and report that the mmchfs command has failed.The FS will retain its old name and still be usable.
fixed a memory overwritten problem caused by uninitialized string copying in mmfs_dm_query_session().
Update restripe code to better handle both replica in same failure group after disk usage and failure group were changed via mmchdisk.
Fix is recommended on all systems in which the disk enclosure supports slot power control. Currently, this includes only the P7IH.
Prevent mmimgbackup from accepting --image option with a value that begins with an absolute path name such as /gpfs/ as this option is meant to only permit a file base name or prefix. Not for specifying the output directory.
Fixed daemon failure on a cluster not configured with the ClusterConfiguration Repository (CCR), but which include Tie-Breaker disks which havea physical sector size of 4096 bytes.
Fix AIO code path to properly manage vfs user count. Incorrect vfs user count could prevent file system quiesce and cause some GPFS commands (ex. snapshot commands) to fail.
Improve handling of unexpectedly corrupted directory blocks.
This fix is recommended for GPFS Native RAID systems running on the AIX operating system. It has no effect on Linux.
Fix inode dealloc issue in AFM LU mode.
Fix node failure recovery code to prevent a possible GPFS daemon assert. This problem could occur if file system manager node fails in the middle of restripe.
Fixed the startup code which will handle the locks in previous uncleaned mmshutdown better.
during mmcrfs, the client doesn't correctly parse the reply for 'nsdMsgConvertFormat', which causes the client failing to move on to the next backup nsd server.
Avoid daemon assert when rejecting illegal mmcrsnapshot commands.
Fix mmfsck assert "More than 22 minutes searching for a free buffer in the pagepool"
fcntl revokes may block unmount under appropriate loads
Fix a problem introduced by the fix for APAR IV56220 that will cause NSD server hangs/failures or possibly data corruption.
Do not log fsstruct errors when offline fsck is running.
When performing automatic disk recovery, take into account the value of defaultHelperNodes when initiating file system restripe operations.
Add additional checks to ensure the uniqueness of certain node attributes.
Fixed assert in setDataValid becuase blockOffset larger than the start offset.
Fix repair code to always performance rebalance for none FPO file system as requested by user.
Prevented I/O from being started after a node has lost contact with the cluster.
Fix bug introduced in GPFS 3.4 PTF30 and GPFS 3.5 PTF20 where mmlsnsd -X doesn't display persistent reserve information of the disk.
Fix code used to check user provided buffer for NULL where it can cause writev() to incorrectly fail with EINVAL. Both readv()/writev() could be affected.
This update addresses the following APARs: IV63359 IV63464 IV63467 IV63469 IV63471 IV63516 IV63518 IV63824 IV63878 IV63882 IV63887 IV63889 IV63894 IV63901 IV63903 IV63911 IV63917 IV63927 IV64598 IV64862.

Problems fixed in GPFS 4.1.0.2 [August 4, 2014]

Add tsm server selection option and improve messages.
write(fd,NULL,bs) gives rc -1 and inconsistent behavior Added a check in code to validate if user provided buffer is NULL. If user provided buffer for rea/write system call is NULL than error is returned much earlier in code.
Fix various problems with RDMA reconnection.
Fix a rare case live lock which can happen when FPO file system is in low space situation.
Fix two integer overflow problems of GPFS block map allocation module which caused by adding larger disk into existing file system. The problem can lead to block lost and data corruption.
Avoid very rare log recovery failure after restripe of snapshot files.
Prevent GPFS file system program mmfsd crash on a GNR/GSS system while deleting a log vdisk.
Fix a problem in locking code that can cause GPFS daemon assert under certain rare race condition. The chance is slightly higher under 4.1.
Prevent file system errors in the face of too many disk errors.
Offline fsck fileset 0 record false positive on v3.3 and older filesystem.
Fix a defect in the fileset snapshot restore tool when the tool tries to restore attributes of directories which they have been deleted after we create fileset snapshot.
Apply if you see tsapolicy failing immediately after a helper node goes down.
Eliminate FSSTRUCT errors from occuring during image restore process. Prevent gpfsLookup() common function from proceeding if stripe group is being image restored.
Fix a node crash defect in gpfs_ireaddirx GPFS API when we use it to list changed dentry for a directory which has data in inode.
Improved stability of mmfsadm dump tscomm.
Install this fix if you have non-standard enclosure / drawer hardware not found in GSS systems.
Fix a defect in the fileset snapshot restore tool when it tries to restore clone file which has been deleted after we create snapshot.
Ignore Grace msg on nodes that do not support Ganesha.
Fixed hung problem due ro lock overflow.
Fix a problem where user registered callback is unexpectedly invoked when using mount instesad of mmmount.
Fix a generation number mismatch defect when we create fileset in GPFS secvm file system.
Fixed a race condition that could lead to an assertion failure when mmpmon is used.
Fixed Assert 'filesetP->hasInodeSpace == 0' in offline fsck.
Fixed problem in multi acquire and release with FGDL.
When there is a GPFS failure return EUNATCH to Ganesha.
Fileset snapshot restore tool restores dir attributes more effectively.
Without this fix a setup with 4 or more drawers in an enclosure may not be able to survive the loss of the enclosure even though mmlsrecoverygruop -L states that disk group fault tolerance can survive an enclosure loss.
Fix a defect that the restore tool cannot sync the restoring fileset when the file system manager node of the restoring fileset is running in GPFS 4.1.0.0 and the restore command is running in a node which runs upper version.
Fixed online fsck assert after block allocation map export.
Must make sure that all the interfaces are enabled.
Fixed Ganesha not using right interface in RHEL6.5.
Fix GPFS_FCNTL_GET_DATABLKDISKIDX fcntl API to return location info of pre allocated block correctly.
clear sector 0 and last 4k bytes of the disk before it is created as NSD to prevent accidental GPT table recovery by uEFI driver.
Fix a race condition problem in fileset snapshot restore tool when it tries to restore extended attributes for a directory.
When GPFS kernel module is loaded on Linux, look up dependent symbols on demand.
Fix stale mount handling in case of home mount issues.
Fixed problem in scanForFileset when sgmgr resigns while the scan is in progress.
Fixed problem where the GPFS daemon may terminate abnormally while performing encryption key rewrap (mmapplypolicy command with "CHANGE ENCRYPTION KEYS" rule). Fixed problem where mmrestorefs -j on an encrypted file system may resultin the file system being unmounted.
Prevent multiple eviction processes from running simultaneously.
Assert in Deque after gracePeriodThread runs.
Update mmchmgr to pick the best candidate as new filesystem manager when user did not specify a new manager node.
Fix a memory leak in the GPFS daemon associated with Events Exporter, mmpmon,and SNMP support.
In GPFS systems employing GPFS Native RAID, there was a situation in which failover and failback, and disk replacement operations could hang, requiring GPFS to be restarted on the node to clear the hang. Fix is extremely low risk and highly recommended.
Fix mmsetquota bug that returns invalid argument if a stanza contain fileset attribute along with type=FILESET.
Fix deadlock if fs panics during E_IO err processing.
mmdeldisk is blocked while phase3 recovery is doing deferred deletions. It is enough to wait until log recovery is done.
Ensure SQL migration is done on GSS nodes only.
Use maxLogReplicas instead of defaultMaxMetadataReplicas to calcuate the number of new log items when a new stripped log is added.
Limit PORTMAP inactive failure due to DNS busy.
Ensure that vdisks are automatically scrubbed periodically.
Initialize the fromFsP to NULL in openArch() to guard against ever calling gpfs_free_fssnaphandle() with a bad argument. Add an informative message to look for the an error log in /tmp when the file writer pipeline is broken.
Correct the multi-release table to avoid releasing fcntl tokens prematurely.
Fixed race condition between two threads trying to become metanode at the same time.
Ensured not to create file if it already exists for NFS when Ganesha is running.
Fixed a typo in in removeOpenFileFrombgdList function that caused sig 11.
Fix code to prevent potential GPFS daemon assert during log recovery. This problem only occurs when filesystem is 4.1 format with 4K alignment enabled (4K inode size, etc). Data replication has to be enabled and direct IO been used for write with buffer size that is not 4K aligned.
This update addresses the following APARs: IV61626 IV61628 IV61630 IV61655 IV61988 IV61991 IV61995 IV62043 IV62091 IV62215 IV62243 IV62418.

Problems fixed in GPFS 4.1.0.1 [June 06, 2014]

Fix thread-safe problem in dumping GPFS daemon threads backtrace.
Fixes a problem with fsck repair of deleted root directory inode of independent filesets.
Fixed a problem in clusters configured for secure communications (cipherListconfiguration variable containing a cipher other than AUTHONLY) which may cause communications between nodes to become blocked.
After a file system is panicked, new lock range request will notbe accepted.
This fix only affects customers running GNR/GSS on Linux, and who have in the past misconfigured their GNR servers by turning the config parameter "numaMemoryInterleave" off, and who experienced IO errors on Vdisks as a result of that misconfiguration. These IO errors can potentially corrupt in-memory metadata of the GNR/GSS server, which can lead to data loss later on. This fix provides a tool that can be used to locate and repair such corruption.
Remove mmchconfig -N restrictions for aioWorkerThreads and enableLinuxReplicatedAio.
Fixed problem when reading a clonde child from a snapshot
Fixed a rare race condition causing the assert when two threads are attempting to do a metanode operation at the same time whilethe node is in the process of becoming a metanode.
Fixed a deadlock in a complicated scenario involving restripe,token revoke and exceeding file cache limit.
Fixed race between log recovery and mnodeResign thread
E_VALIDATE errors in the aclFile after node failure
Deal with stress condition where mmfsd was running out of threads
Fix a problem in log recovery that would cause it to fail when replaying a directory insert record. The error only occurs for filesystems in version 4.1 format, where the hash value of the file name being inserted is the same as an existing file in the directory. The problem is also dependent on the length of the file name, and only happens if the system crashes after the log record is committed, but before the directory contents are flushed.
Fixed the problem that was caused by a hole in the cleanBufferafter the file system panicked.
Close a hole that fileset snapshot restore tool (mmrestorefs -j) may cannot restore changed data for a clone child file.
Fix a rare assert which happens under low disk space situation
Fixed deadlock during mmap pagein
Fixed the problem of excessive RPCs to get indirect blocks and the problemof metanode lock starvation involving a huge sized sparse file.
A problem has been fixed where the GPFS daemon terminates abnormallywhen direct I/O and vector I/O (readv/writev) is used on encrypted files,and the data is replicated, or the data must go through an NSD server.
Fix a potential deadlock when selinux is enabled and FS is dmapi managed.
Close a hole that fileset snapshot restore tool (mmrestorefs -j) may cannot restore a snapshot in a race condition that one restore thread is deleting a file but another restore thread is also trying to get file attributes for this file.
Fixed a kernel oops that caused by a race in multiple NFS readson the same large file.
mmchfirmware command will avoid accessing non-existent disk path.
Fix a directory generation mismatch problem in an encrypted secvm file system.
shutdown hangs in the kernel trying to acquire revokeLock
Apply at your convenience. Even if you hit this bug, an equivalent cleanup is completed later in the command execution.
improved stability of daemon-to-daemon communications when cipherList is set to a real cipher (i.e. not AUTHONLY).
The serial number of physical disks is now recorded in the GNR event log, and displayed in the mmlspdisk command.
GNR on AIX allow only 32K segment.
Fixes a problem with fsck repair of corrupt root directory inode
mmbackup tricked by false TSM success messages Mmbackup can be fooled by TSM output when dsmc decides to roll-back a transaction of multiple files being backed up. When the TSM server runs out of data storage space, the current transaction which may hold many files will be rolled back and re-tried with each file separately. The failure of a file to be backed up in this case was not detected because the earlier message from dsmc contained "Normal File --> [Sent]" though it was later rolled back. Fixes in tsbuhelper now detect the failure signature "** Unsuccessfull **" string and instead of simply ignoring these now will revert the changes in the shadow DB for the matching record(s). Hash table keeps track of last change in each record already, so reverting is now a legal state transition for hashed records. Reorganized some debug messages and streamlined some common code to work better. Now find 'failed' string to issue reversion updates as well. Fixed pattern matching in tsbackup33.pl to properly display all "ANS####" messages.
Fix RO cache i/o error if mounting fs in ro mode.
Don't release mutex if daemon death.
Fix the path buffer length calculation to return the correct length for dm_handle_to_path() functions.
Fix bug in mmauth that may cause duplicate configure entries and node numbermismatch in configure file.
Fix a problem with creating directory if the parent directory has default POSIX ACL.
mmbackup fails to read hex env values mmbackup debug values, progress reporting, and possibly other user settings may be presented in decimal or hex, especially the bit-mapped progress and debugging settings. Perl doesn't always interpret the hex values correctly unless converted with the oct() function.
Correct an NLS-related problem with mmchdisk and similar commands.
This update addresses the following APARs: IV60187 IV60468 IV60469 IV60471 IV60475 IV60478 IV60543 IV60863 IV60864.

Product/Component Name:	Platform:	Fix:
IBM Spectrum Scale	Windows 2008	Spectrum_Scale_Express-4.1.1.7-x86_64-Windows

Readme and Release notes for release 4.1.1.7 IBM Spectrum Scale 4.1.1.7 Spectrum_Scale_Express-4.1.1.7-x86_64-Windows Readme