Fix (APAR): PI15453 Status: Fix Release: 7.0.0.31 Operating System: Windows Supersedes Fixes: CMVC Defect: xxxxxx Byte size of APAR: 190278 Date: 2014-05-19 Abstract: aio tcp channel could hang on Microsoft Windows operating systems Description/symptom of problem: PI15453 resolves the following problem: ERROR DESCRIPTION: Due to a problem introduced by APAR PM95795, threads may hang and an application server may stop responding to requests on version V7.0.0.31 or V8.0.0.8. This can happen when Asynchronous I/O (AIO) is enabled, which it is by default, and it can only occur on Windows operating systems. . Javacores and the SystemOut.log (WSVR0605W messages) may show many threads hung with a stack trace similar to the following: . Java callstack: at com/ibm/io/async/AsyncLibrary.terminateIOCB(AsyncLibrary.java:78 8(Compiled Code)) at com/ibm/io/async/AsyncSocketChannel.close(AsyncSocketChannel.jav a:314(Compiled Code)) at com/ibm/ws/tcp/channel/impl/AioSocketIOChannel.close(AioSocketIO Channel.java:435(Compiled Code)) at com/ibm/ws/tcp/channel/impl/TCPConnLink.destroy(TCPConnLink.java :530(Compiled Code)) at com/ibm/wsspi/channel/base/OutboundConnectorLink.close(OutboundC onnectorLink.java:50(Compiled Code)) at com/ibm/ws/tcp/channel/impl/TCPConnLink.close(TCPConnLink.java:5 06(Compiled Code)) at com/ibm/ws/http/channel/inbound/impl/HttpInboundLink.close(HttpI nboundLink.java:785(Compiled Code)) at com/ibm/ws/http/channel/inbound/impl/HttpICLReadCallback.error(H ttpICLReadCallback.java:141) at com/ibm/ws/tcp/channel/impl/AioReadCompletionListener.futureComp leted(AioReadCompletionListener.java:220(Compiled Code)) at com/ibm/io/async/AbstractAsyncFuture.invokeCallback(AbstractAsyn cFuture.java:217(Compiled Code)) at com/ibm/io/async/AsyncChannelFuture.fireCompletionActions(AsyncC hannelFuture.java:161(Compiled Code)) at com/ibm/io/async/AbstractAsyncFuture.completed(AbstractAsyncFutu re.java:170) at com/ibm/io/async/ResultHandler.complete(ResultHandler.java:213(C ompiled Code)) at com/ibm/io/async/ResultHandler.runEventProcessingLoop(ResultHand ler.java:775(Compiled Code)) at com/ibm/io/async/ResultHandler$2.run(ResultHandler.java:905(Comp iled Code)) at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1660(Compi led Code)) . These threads are blocked by another thread with the following stack trace: . Java callstack: at com/ibm/io/async/AsyncLibrary.aio_cancel2(Native Method) at com/ibm/io/async/AsyncLibrary.cancel2(AsyncLibrary.java:462) at com/ibm/io/async/AbstractAsyncChannel.cancel(AbstractAsyncChanne l.java:287) at com/ibm/io/async/AsyncChannelFuture.cancel(AsyncChannelFuture.ja va:114) at com/ibm/io/async/AsyncSocketChannelHelper$tCallback.timerTrigger ed(AsyncSocketChannelHelper.java:116) at com/ibm/io/async/Timer.checkForTimeouts(Timer.java:426(Compiled Code)) at com/ibm/io/async/Timer.run(Timer.java:241(Compiled Code)) LOCAL FIX: AIO can be disabled to prevent the problem. PROBLEM SUMMARY USERS AFFECTED: IBM WebSphere Application Server Full Profile Versions 7.0.0.31 and 8.0.0.8 users of AIO and on Microsoft Windows operating systems PROBLEM DESCRIPTION: A thread could hang when processing a timeout RECOMMENDATION: None A thread could hang when processing a possible timeout of a TCP read/write operation due to thread synchronization issues. This could occur, when trying to confirm a read/write had been successfully cancelled and the operation could be truly declared a timeout. This problem was introduced with PM95795 PROBLEM CONCLUSION: To avoid the hang, polling code was added to the TCP AIO code to monitor the state of the cancel operation The fix for this APAR is currently targeted for inclusion in fix packs 7.0.0.33, 8.0.0.9 and 8.5.5.2. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980 Directions to apply fix: Fix applies to Editions: Release 7.0 _X_ Application Server (Express or BASE) _X_ Network Deployment (ND) __ Edge Components __ Developer Install Fix to all WebSphere installations unless special instructions are included below. Special Instructions: None NOTE: The user must: * Logged in with the same authority level when unpacking a fix, fix pack or refresh pack. * Be at V7.0.0.0 or newer of the Update Installer. Certain iFixes may require a newer version of the Update Installer and the Update Installer will inform you during the installation process if a newer version is required. This can be checked by reviewing the level of the Update Installer in file /updateinstaller/version.txt. The Update Installer can be downloaded from the following link: http://www.ibm.com/support/docview.wss?rs=180&uid=swg21205991 1) If your iFix is delivered as a single file with a .pak extension, Copy the .pak file directly to the maintenance directory. If your iFix is delivered as a single file with a .zip extension, unzip the file into the maintenance directory. 2) Shutdown WebSphere Application Server. Manually execute setupCmdLine.bat in Windows or . ./setupCmdLine.sh in Unix from the WebSphere instance that maintenance is being applied to. 3) For IBM i users, use the update command to install and uninstall the interim fix. The IBM Information Center can provide additional details, if needed, on the use of this command. http://www14.software.ibm.com/webapp/wsbroker/redirect?version=compass&product=was-nd-iseries&topic=rins_update. For non-IBM i users, launch the Update Installer and click the Next button on the Welcome page. 4) Enter the directory path of the installation location of the WebSphere product you want to update, and click the Next button. 5) Select the "Install maintenance package" operation and click the Next button. 6) Enter the directory path of your maintenance directory where you have the maintenance packages (.pak files) and click the Next button. 7) The Available Maintenance Package to Install page should list all maintenance packages (.pak files) that it finds in the directory path provided in the previous step. The Update Installer will select the correct maintenance packages based on your system configuration and will not allow an invalid combination to be installed. Please keep the Update Installer recommendations and click the Next button and continue with the installation of the maintenance package. To determine why some maintenance packages have been identified as not applicable, see description in log found in /logs/tmp*/updatelogs.txt 8) For all platforms except Windows. In pre-install summary panel, use the "verify permission" feature to verify the user has permissions to apply updates to files associated with the selected maintenance. Correct any file permissions before clicking next to start the install. 9) Restart WebSphere Application Server. Directions to remove fix: NOTE: * The user must have Administrative rights in Windows, or be the Actual Root User in a UNIX environments. * FIXES MUST BE REMOVED IN THE ORDER THEY WERE APPLIED * DO NOT REMOVE A FIX UNLESS ALL FIXES APPLIED AFTER IT HAVE FIRST BEEN REMOVED * YOU MAY REAPPLY ANY REMOVED FIX Example: If your system has fix1, fix2, and fix3 applied in that order and fix2 is to be removed, fix3 must be removed first, fix2 removed, and fix3 re-applied. 1) Shutdown WebSphere Application Server. Manually execute setupCmdLine.bat in Windows or . ./setupCmdLine.sh in Unix from the WebSphere instance that uninstall is being run against. 2) Start Update Installer 3) Enter the installation location of the WebSphere product you want to remove the fix. 4) Select "Uninstall maintenance package" operation. 5) Enter the file name of the maintenance package to uninstall (PKxxxxx.pak). 6) UnInstall maintenance package. 7) Restart WebSphere Directions to re-apply fix: 1) Shutdown WebSphere Application Server. 2) Follow the Fix instructions to apply the fix. 3) Restart WebSphere Application Server. Additional Information: