[jboss-jira] [JBoss JIRA] Closed: (JBAS-4459) Hitting Ping-Pong Bug in 4.0.4.GA

Hitting Ping-Pong Bug in 4.0.4.GA --------------------------------- Key: JBAS-4459 URL: http://jira.jboss.com/jira/browse/JBAS-4459 Project: JBoss Application Server Issue Type: Bug Security Level: Public(Everyone can see) Components: JMS service Affects Versions: JBossAS-4.0.4.GA Environment: OS: Windows XP 5.1 (x86) Reporter: Rahul Sule Assigned To: Adrian Brock I was looking the Ping-Pong bug (JBAS-1489) that was fixed sometime back. It is now fixed but I am hitting the same issue. We were hitting the same issue with Jobss-3.2.7 also where the stack was a bit different. Now with Jboss-3.0.4.GA we are hitting it again and "at org.jboss.mq.Connection$PingTask.run" in stack trace makes me believe that this is same issue surfaced in Jboss-4.0.4.GA. We have the exception listeners set as suggested in the workaround before. Following is what I see in web-console page. ==================================================== Version - Version: 4.0.4GA(build: CVSTag=JBoss_4_0_4_GA date=200605151000) Version Name: Zion Built on: May 15 2006 Hardware - #CPU: 2 OS: Windows XP 5.1 (x86) JVM Environment - Free Memory: 810 MB Max Memory: 1018 MB Total Memory: 1018 MB #Threads: 118 JVM Version: 1.4.2_10-b03 (Sun Microsystems Inc.) JVM Name: Java HotSpot(TM) Server VM ==================================================== Following is the stack I get ono server stuckup ==================================================== 2007-05-31 19:32:25,091 INFO [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX] (ExceptionListener Connection@8451076[token=ConnectionToken:ID:20/d24f31e103c633d6b680a78e3e7b7e4c rcvstate=STARTED]) Detected JMS exception. org.jboss.mq.SpyJMSException: No pong received; - nested throwable: (java.io.IOException: ping timeout.) at org.jboss.mq.Connection$PingTask.run(Connection.java:1277) at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: ping timeout. ... 3 more 2007-05-31 19:32:25,091 WARN [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX cache] (ExceptionListener Connection@1331620[token=ConnectionToken:ID:17/b765cf139828d66b4f12cbe272d4fdf5 rcvstate=STARTED]) ---------- Detected JMS Exception (in AbstractCache) ---------------- org.jboss.mq.SpyJMSException: No pong received; - nested throwable: (java.io.IOException: ping timeout.) at org.jboss.mq.Connection$PingTask.run(Connection.java:1277) at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: ping timeout. ... 3 more 2007-05-31 19:32:25,091 INFO [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX cache] (ExceptionListener Connection@1331620[token=ConnectionToken:ID:17/b765cf139828d66b4f12cbe272d4fdf5 rcvstate=STARTED]) Reconnecting JMS resources... 2007-05-31 19:32:25,091 INFO [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX] (ExceptionListener Connection@33292728[token=ConnectionToken:ID:22/db775d643d19a48d8e1532dfa41bbe7c rcvstate=STARTED]) Detected JMS exception. org.jboss.mq.SpyJMSException: No pong received; - nested throwable: (java.io.IOException: ping timeout.) at org.jboss.mq.Connection$PingTask.run(Connection.java:1277) at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: ping timeout. ... 3 more 2007-05-31 19:34:32,978 INFO [XXXXXXXXXXXXXXXXX.Application] (ExceptionListener Connection@1276382[token=ConnectionToken:ID:19/6de512f89fdd63fec782954f8d783121 rcvstate=STARTED]) Detected JMS exception. org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketTimeoutException: Read timed out) at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:78) at org.jboss.mq.Connection.asynchFailure(Connection.java:420) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:174) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:440) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:372) at java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:79) at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.refill(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.read(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.readByte(Unknown Source) at java.io.ObjectInputStream.readByte(Unknown Source) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:317) ... 1 more 2007-05-31 19:34:55,739 WARN [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX cache] (ExceptionListener Connection@1331620[token=ConnectionToken:ID:17/b765cf139828d66b4f12cbe272d4fdf5 rcvstate=STARTED]) Error while establishing JMS connection. org.jboss.mq.SpyJMSException: Cannot authenticate user; - nested throwable: (java.net.SocketTimeoutException: Read timed out) at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:78) at org.jboss.mq.SpyJMSException.rethrowAsJMSException(SpyJMSException.java:63) at org.jboss.mq.Connection.authenticate(Connection.java:1069) at org.jboss.mq.Connection.<init>(Connection.java:252) at org.jboss.mq.Connection.<init>(Connection.java:323) at org.jboss.mq.SpyConnection.<init>(SpyConnection.java:116) at org.jboss.mq.SpyConnectionFactory.internalCreateConnection(SpyConnectionFactory.java:137) at org.jboss.mq.SpyConnectionFactory.createTopicConnection(SpyConnectionFactory.java:118) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.connectJMS(AbstractCache.java:401) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.reconnectJMS(AbstractCache.java:376) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.onException(AbstractCache.java:720) at org.jboss.mq.Connection$ExceptionListenerRunnable.run(Connection.java:1320) at java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read1(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:89) at java.io.ObjectInputStream$PeekInputStream.read(Unknown Source) at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source) at java.io.ObjectInputStream.readStreamHeader(Unknown Source) at java.io.ObjectInputStream.<init>(Unknown Source) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:304) ... 1 more 2007-05-31 19:34:55,739 WARN [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX cache] (ExceptionListener Connection@1331620[token=ConnectionToken:ID:17/b765cf139828d66b4f12cbe272d4fdf5 rcvstate=STARTED]) Error while reconnecting to JMS. org.jboss.mq.SpyJMSException: Cannot authenticate user; - nested throwable: (java.net.SocketTimeoutException: Read timed out) at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:78) at org.jboss.mq.SpyJMSException.rethrowAsJMSException(SpyJMSException.java:63) at org.jboss.mq.Connection.authenticate(Connection.java:1069) at org.jboss.mq.Connection.<init>(Connection.java:252) at org.jboss.mq.Connection.<init>(Connection.java:323) at org.jboss.mq.SpyConnection.<init>(SpyConnection.java:116) at org.jboss.mq.SpyConnectionFactory.internalCreateConnection(SpyConnectionFactory.java:137) at org.jboss.mq.SpyConnectionFactory.createTopicConnection(SpyConnectionFactory.java:118) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.connectJMS(AbstractCache.java:401) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.reconnectJMS(AbstractCache.java:376) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.onException(AbstractCache.java:720) at org.jboss.mq.Connection$ExceptionListenerRunnable.run(Connection.java:1320) at java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read1(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:89) at java.io.ObjectInputStream$PeekInputStream.read(Unknown Source) at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source) at java.io.ObjectInputStream.readStreamHeader(Unknown Source) at java.io.ObjectInputStream.<init>(Unknown Source) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:304) ... 1 more 2007-05-31 19:34:55,739 WARN [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX cache] (ExceptionListener Connection@1331620[token=ConnectionToken:ID:17/b765cf139828d66b4f12cbe272d4fdf5 rcvstate=STARTED]) ---------- End of handling JMS exception (in AbstractCache) ---------------- 2007-05-31 19:35:03,775 ERROR [XXXXXXXXXXXXXXXXXXApplication] (ExceptionListener Connection@33292728[token=ConnectionToken:ID:22/db775d643d19a48d8e1532dfa41bbe7c rcvstate=STARTED]) Got JMSException from JMS provider while closing TS reply receiver: org.jboss.mq.SpyJMSException: Cannot unsubscribe to this destination; - nested throwable: (java.io.IOException: Client is not connected) Vendor specific error code: null Linked exception. org.jboss.mq.SpyJMSException: Cannot unsubscribe to this destination; - nested throwable: (java.io.IOException: Client is not connected) at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:78) at org.jboss.mq.SpyJMSException.rethrowAsJMSException(SpyJMSException.java:63) at org.jboss.mq.Connection.removeConsumer(Connection.java:899) at org.jboss.mq.SpySession.removeConsumer(SpySession.java:968) at org.jboss.mq.SpyMessageConsumer.close(SpyMessageConsumer.java:563) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.disconnectJMS(Application.java:1191) at XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.onException(Application.java:3959) at org.jboss.mq.Connection$ExceptionListenerRunnable.run(Connection.java:1320) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Client is not connected at org.jboss.mq.il.uil2.SocketManager.internalSendMessage(SocketManager.java:265) at org.jboss.mq.il.uil2.SocketManager.sendMessage(SocketManager.java:221) at org.jboss.mq.il.uil2.UILServerIL.unsubscribe(UILServerIL.java:551) at org.jboss.mq.Connection.removeConsumer(Connection.java:893) ... 6 more 2007-05-31 19:35:03,775 WARN [org.jboss.mq.Connection] (UIL2.SocketManager.ReadTask#9 client=3.187.30.55:8093) Connection failure, already in the exception listener org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketTimeoutException: Read timed out) at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:78) at org.jboss.mq.Connection.asynchFailure(Connection.java:420) at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:174) at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:440) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:372) at java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:79) at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.refill(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.read(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.readByte(Unknown Source) at java.io.ObjectInputStream.readByte(Unknown Source) at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:317) ... 1 more ====================================================