[JBoss JIRA] Created: (JBMESSAGING-702) Dead Lock condition on MessageCallbackHandler when failover is being executed
10:22 a.m.
Dead Lock condition on MessageCallbackHandler when failover is being executed
-----------------------------------------------------------------------------
Key: JBMESSAGING-702
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-702
Project: JBoss Messaging
Issue Type: Bug
Reporter: Clebert Suconic
Assigned To: Clebert Suconic
If a failover happens when an auto ACK, is returned from MessageCallbackHandler (look at
the thread dump) a dead lock might happen:
LocalThreadConsumer-1" prio=1 tid=0x8b9cf568 nid=0x228e in Object.wait()
[0x8a66f000..0x8a66feb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
- locked <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.jboss.jms.client.container.ValveAspect.invoke(ValveAspect.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledgeDelivery(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.ackDelivery(SessionAspect.java:453)
at org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:283)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect7.invoke(SessionAspect7.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at
org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:419)
- locked <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
at
org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect25.invoke(ReceiverAspect25.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
at
org.jboss.test.messaging.jms.clustering.ValveTest$LocalThreadConsumer.run(ValveTest.java:94)
"Thread-6" daemon prio=1 tid=0x08183840 nid=0x22d2 waiting for monitor entry
[0x8aefc000..0x8aefd030]
at
org.jboss.jms.client.remoting.MessageCallbackHandler.copyState(MessageCallbackHandler.java:500)
- waiting to lock <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.HAAspect.handleFailoverOnConsumer(HAAspect.java:605)
at org.jboss.jms.client.container.HAAspect.performClientSideFailover(HAAspect.java:451)
at org.jboss.jms.client.container.HAAspect.handleConnectionFailure(HAAspect.java:337)
at
org.jboss.jms.client.container.ValveAspect.handleConnectionFailure(ValveAspect.java:202)
at
org.jboss.jms.client.container.ValveAspect$ConnectionFailureListener.handleConnectionException(ValveAspect.java:241)
at
org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:83)
at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:139)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:29 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-702) Dead Lock condition on MessageCallbackHandler when failover is being executed
[ http://jira.jboss.com/jira/browse/JBMESSAGING-702?page=comments#action_12... ]
Clebert Suconic commented on JBMESSAGING-702:
---------------------------------------------
HAStressTest just timed out on cruisecontrol because of this issue.
Probably servers were still running (hang maybe?) after this, what caused problems on the
testsuite.
http://cruisecontrol.jboss.com/cc/artifacts/jms-head-testsuite-mysql/2006...
Dead Lock condition on MessageCallbackHandler when failover is being
executed
-----------------------------------------------------------------------------
Key: JBMESSAGING-702
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-702
Project: JBoss Messaging
Issue Type: Bug
Reporter: Clebert Suconic
Assigned To: Clebert Suconic
If a failover happens when an auto ACK, is returned from MessageCallbackHandler (look at
the thread dump) a dead lock might happen:
LocalThreadConsumer-1" prio=1 tid=0x8b9cf568 nid=0x228e in Object.wait()
[0x8a66f000..0x8a66feb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
- locked <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.jboss.jms.client.container.ValveAspect.invoke(ValveAspect.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledgeDelivery(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.ackDelivery(SessionAspect.java:453)
at
org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:283)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect7.invoke(SessionAspect7.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at
org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:419)
- locked <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
at
org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect25.invoke(ReceiverAspect25.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
at
org.jboss.test.messaging.jms.clustering.ValveTest$LocalThreadConsumer.run(ValveTest.java:94)
"Thread-6" daemon prio=1 tid=0x08183840 nid=0x22d2 waiting for monitor entry
[0x8aefc000..0x8aefd030]
at
org.jboss.jms.client.remoting.MessageCallbackHandler.copyState(MessageCallbackHandler.java:500)
- waiting to lock <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.HAAspect.handleFailoverOnConsumer(HAAspect.java:605)
at org.jboss.jms.client.container.HAAspect.performClientSideFailover(HAAspect.java:451)
at org.jboss.jms.client.container.HAAspect.handleConnectionFailure(HAAspect.java:337)
at
org.jboss.jms.client.container.ValveAspect.handleConnectionFailure(ValveAspect.java:202)
at
org.jboss.jms.client.container.ValveAspect$ConnectionFailureListener.handleConnectionException(ValveAspect.java:241)
at
org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:83)
at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:139)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:32 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-702) Dead Lock condition on MessageCallbackHandler when failover is being executed
[ http://jira.jboss.com/jira/browse/JBMESSAGING-702?page=comments#action_12... ]
Clebert Suconic commented on JBMESSAGING-702:
---------------------------------------------
On my local copy I have fixed the issue by removing the mainLock.
As this is only activated from a failover, this should be okay, since failover will be
holding a writeLock.
public void copyState(MessageCallbackHandler newHandler)
{
//synchronized (mainLock)
//{
this.consumerID = newHandler.consumerID;
this.consumerDelegate = newHandler.consumerDelegate;
this.sessionDelegate = newHandler.sessionDelegate;
this.buffer.clear();
//}
}
Dead Lock condition on MessageCallbackHandler when failover is being
executed
-----------------------------------------------------------------------------
Key: JBMESSAGING-702
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-702
Project: JBoss Messaging
Issue Type: Bug
Reporter: Clebert Suconic
Assigned To: Clebert Suconic
If a failover happens when an auto ACK, is returned from MessageCallbackHandler (look at
the thread dump) a dead lock might happen:
LocalThreadConsumer-1" prio=1 tid=0x8b9cf568 nid=0x228e in Object.wait()
[0x8a66f000..0x8a66feb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
- locked <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.jboss.jms.client.container.ValveAspect.invoke(ValveAspect.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledgeDelivery(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.ackDelivery(SessionAspect.java:453)
at
org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:283)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect7.invoke(SessionAspect7.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at
org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:419)
- locked <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
at
org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect25.invoke(ReceiverAspect25.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
at
org.jboss.test.messaging.jms.clustering.ValveTest$LocalThreadConsumer.run(ValveTest.java:94)
"Thread-6" daemon prio=1 tid=0x08183840 nid=0x22d2 waiting for monitor entry
[0x8aefc000..0x8aefd030]
at
org.jboss.jms.client.remoting.MessageCallbackHandler.copyState(MessageCallbackHandler.java:500)
- waiting to lock <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.HAAspect.handleFailoverOnConsumer(HAAspect.java:605)
at org.jboss.jms.client.container.HAAspect.performClientSideFailover(HAAspect.java:451)
at org.jboss.jms.client.container.HAAspect.handleConnectionFailure(HAAspect.java:337)
at
org.jboss.jms.client.container.ValveAspect.handleConnectionFailure(ValveAspect.java:202)
at
org.jboss.jms.client.container.ValveAspect$ConnectionFailureListener.handleConnectionException(ValveAspect.java:241)
at
org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:83)
at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:139)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:37 a.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-702) Dead Lock condition on MessageCallbackHandler when failover is being executed
[ http://jira.jboss.com/jira/browse/JBMESSAGING-702?page=all ]
Clebert Suconic updated JBMESSAGING-702:
----------------------------------------
JBoss Forum Reference:
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=97150&am...
Dead Lock condition on MessageCallbackHandler when failover is being
executed
-----------------------------------------------------------------------------
Key: JBMESSAGING-702
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-702
Project: JBoss Messaging
Issue Type: Bug
Reporter: Clebert Suconic
Assigned To: Clebert Suconic
If a failover happens when an auto ACK, is returned from MessageCallbackHandler (look at
the thread dump) a dead lock might happen:
LocalThreadConsumer-1" prio=1 tid=0x8b9cf568 nid=0x228e in Object.wait()
[0x8a66f000..0x8a66feb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
- locked <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.jboss.jms.client.container.ValveAspect.invoke(ValveAspect.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledgeDelivery(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.ackDelivery(SessionAspect.java:453)
at
org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:283)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect7.invoke(SessionAspect7.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at
org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:419)
- locked <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
at
org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect25.invoke(ReceiverAspect25.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
at
org.jboss.test.messaging.jms.clustering.ValveTest$LocalThreadConsumer.run(ValveTest.java:94)
"Thread-6" daemon prio=1 tid=0x08183840 nid=0x22d2 waiting for monitor entry
[0x8aefc000..0x8aefd030]
at
org.jboss.jms.client.remoting.MessageCallbackHandler.copyState(MessageCallbackHandler.java:500)
- waiting to lock <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.HAAspect.handleFailoverOnConsumer(HAAspect.java:605)
at org.jboss.jms.client.container.HAAspect.performClientSideFailover(HAAspect.java:451)
at org.jboss.jms.client.container.HAAspect.handleConnectionFailure(HAAspect.java:337)
at
org.jboss.jms.client.container.ValveAspect.handleConnectionFailure(ValveAspect.java:202)
at
org.jboss.jms.client.container.ValveAspect$ConnectionFailureListener.handleConnectionException(ValveAspect.java:241)
at
org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:83)
at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:139)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:17 p.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-702) Dead Lock condition on MessageCallbackHandler when failover is being executed
[ http://jira.jboss.com/jira/browse/JBMESSAGING-702?page=all ]
Clebert Suconic updated JBMESSAGING-702:
----------------------------------------
Fix Version/s: 1.2.0.Beta2
Affects Version/s: 1.2.0.Beta1
Dead Lock condition on MessageCallbackHandler when failover is being
executed
-----------------------------------------------------------------------------
Key: JBMESSAGING-702
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-702
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.2.0.Beta1
Reporter: Clebert Suconic
Assigned To: Clebert Suconic
Fix For: 1.2.0.Beta2
If a failover happens when an auto ACK, is returned from MessageCallbackHandler (look at
the thread dump) a dead lock might happen:
LocalThreadConsumer-1" prio=1 tid=0x8b9cf568 nid=0x228e in Object.wait()
[0x8a66f000..0x8a66feb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
- locked <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.jboss.jms.client.container.ValveAspect.invoke(ValveAspect.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledgeDelivery(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.ackDelivery(SessionAspect.java:453)
at
org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:283)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect7.invoke(SessionAspect7.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at
org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:419)
- locked <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
at
org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect25.invoke(ReceiverAspect25.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
at
org.jboss.test.messaging.jms.clustering.ValveTest$LocalThreadConsumer.run(ValveTest.java:94)
"Thread-6" daemon prio=1 tid=0x08183840 nid=0x22d2 waiting for monitor entry
[0x8aefc000..0x8aefd030]
at
org.jboss.jms.client.remoting.MessageCallbackHandler.copyState(MessageCallbackHandler.java:500)
- waiting to lock <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.HAAspect.handleFailoverOnConsumer(HAAspect.java:605)
at org.jboss.jms.client.container.HAAspect.performClientSideFailover(HAAspect.java:451)
at org.jboss.jms.client.container.HAAspect.handleConnectionFailure(HAAspect.java:337)
at
org.jboss.jms.client.container.ValveAspect.handleConnectionFailure(ValveAspect.java:202)
at
org.jboss.jms.client.container.ValveAspect$ConnectionFailureListener.handleConnectionException(ValveAspect.java:241)
at
org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:83)
at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:139)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:59 p.m.
New subject: [JBoss JIRA] Closed: (JBMESSAGING-702) Dead Lock condition on MessageCallbackHandler when failover is being executed
[ http://jira.jboss.com/jira/browse/JBMESSAGING-702?page=all ]
Clebert Suconic closed JBMESSAGING-702.
---------------------------------------
Resolution: Done
I run the entire testsuite, and I didn't get any unexpected failures.
Dead Lock condition on MessageCallbackHandler when failover is being
executed
-----------------------------------------------------------------------------
Key: JBMESSAGING-702
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-702
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.2.0.Beta1
Reporter: Clebert Suconic
Assigned To: Clebert Suconic
Fix For: 1.2.0.Beta2
If a failover happens when an auto ACK, is returned from MessageCallbackHandler (look at
the thread dump) a dead lock might happen:
LocalThreadConsumer-1" prio=1 tid=0x8b9cf568 nid=0x228e in Object.wait()
[0x8a66f000..0x8a66feb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
- locked <0x91652838> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.jboss.jms.client.container.ValveAspect.invoke(ValveAspect.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.invokeNext(ClientSessionDelegate$acknowledgeDelivery_N3172014476730533936.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledgeDelivery(ClientSessionDelegate.java)
at org.jboss.jms.client.container.SessionAspect.ackDelivery(SessionAspect.java:453)
at
org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:283)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect7.invoke(SessionAspect7.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
at
org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:419)
- locked <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
at
org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect25.invoke(ReceiverAspect25.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:177)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:68)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
at
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
at
org.jboss.test.messaging.jms.clustering.ValveTest$LocalThreadConsumer.run(ValveTest.java:94)
"Thread-6" daemon prio=1 tid=0x08183840 nid=0x22d2 waiting for monitor entry
[0x8aefc000..0x8aefd030]
at
org.jboss.jms.client.remoting.MessageCallbackHandler.copyState(MessageCallbackHandler.java:500)
- waiting to lock <0x91652338> (a java.lang.Object)
at org.jboss.jms.client.container.HAAspect.handleFailoverOnConsumer(HAAspect.java:605)
at org.jboss.jms.client.container.HAAspect.performClientSideFailover(HAAspect.java:451)
at org.jboss.jms.client.container.HAAspect.handleConnectionFailure(HAAspect.java:337)
at
org.jboss.jms.client.container.ValveAspect.handleConnectionFailure(ValveAspect.java:202)
at
org.jboss.jms.client.container.ValveAspect$ConnectionFailureListener.handleConnectionException(ValveAspect.java:241)
at
org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:83)
at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:139)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6506
days inactive
6506
days old
5 comments
1 participants
participants (1)
-
Clebert Suconic (JIRA)