[jboss-jira] [JBoss JIRA] (JBMESSAGING-1955) Deadlock on failover with client lease

[Doug Grove (JIRA)]

Doug Grove created JBMESSAGING-1955:
---------------------------------------

             Summary: Deadlock on failover with client lease
                 Key: JBMESSAGING-1955
                 URL: https://issues.jboss.org/browse/JBMESSAGING-1955
             Project: JBoss Messaging
          Issue Type: Feature Request
          Components: Messaging Core Distributed Support
    Affects Versions: 1.4.8.SP5
         Environment: JBoss EAP 5.1.2
            Reporter: Doug Grove


JBoss is configured as an 8 node cluster.  This cluster is for JMS messaging only.  Other JBoss instances host MDBs.  When failover occurs in the 8 node cluster, a deadlock is seen on the JBoss instances that host the MDBs.

{quote}
Found one Java-level deadlock:
=============================
"Thread-177":
  waiting to lock monitor 0x4c561e18 (object 0x7c3dfdb0, a java.lang.Object),
  which is held by "Thread-130"
"Thread-130":
  waiting to lock monitor 0x08aea1a4 (object 0x81dde148, a java.lang.Object),
  which is held by "Timer-18"
"Timer-18":
  waiting to lock monitor 0x4c561e18 (object 0x7c3dfdb0, a java.lang.Object),
  which is held by "Thread-130"

Java stack information for the threads listed above:
===================================================
"Thread-177":
	at org.jboss.remoting.Client.removeConnectionListener(Client.java:599)
	- waiting to lock <0x7c3dfdb0> (a java.lang.Object)
	at org.jboss.jms.client.remoting.JMSRemotingConnection.removeConnectionListener(JMSRemotingConnection.java:565)
	- locked <0x7f589740> (a org.jboss.jms.client.remoting.JMSRemotingConnection)
	at org.jboss.jms.client.container.ConnectionAspect.handleClose(ConnectionAspect.java:183)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
	at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:172)
	at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.jms.client.delegate.ClientConnectionDelegate.close(ClientConnectionDelegate.java)
	at org.jboss.jms.client.JBossConnection.close(JBossConnection.java:132)
	at org.jboss.resource.adapter.jms.inflow.JmsActivation.teardownConnection(JmsActivation.java:635)
	at org.jboss.resource.adapter.jms.inflow.JmsActivation.teardown(JmsActivation.java:376)
	at org.jboss.resource.adapter.jms.inflow.JmsActivation.handleFailure(JmsActivation.java:275)
	at org.jboss.resource.adapter.jms.inflow.JmsActivation.onException(JmsActivation.java:312)
	at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:120)
	- locked <0x7f320410> (a java.lang.Object)
	- locked <0x7f5899a0> (a org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener)
	at org.jboss.remoting.ConnectionValidator$3.run(ConnectionValidator.java:524)
"Thread-130":
	at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
	- waiting to lock <0x81dde148> (a java.lang.Object)
	at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
	- locked <0x7c3dfdb0> (a java.lang.Object)
	at org.jboss.remoting.Client.connect(Client.java:1918)
	at org.jboss.remoting.Client.connect(Client.java:737)
	at org.jboss.jms.client.remoting.JMSRemotingConnection$1.run(JMSRemotingConnection.java:374)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.jboss.jms.client.remoting.JMSRemotingConnection.start(JMSRemotingConnection.java:368)
	at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:175)
	at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeTarget(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
	at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:80)
	at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect_z_handleCreateConnectionDelegate_25018031.invoke(StateCreationAspect_z_handleCreateConnectionDelegate_25018031.java)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
	at org.jboss.jms.client.container.ClusteringAspect.handleCreateConnectionDelegate(ClusteringAspect.java:134)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate.createConnectionDelegate(ClientClusteredConnectionFactoryDelegate.java)
	at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:129)
	at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
	at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:84)
	- locked <0x813598d0> (a org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener)
	at org.jboss.remoting.ConnectionValidator$3.run(ConnectionValidator.java:524)
"Timer-18":
	at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
	- waiting to lock <0x7c3dfdb0> (a java.lang.Object)
	at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
	at org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
	- locked <0x81dde148> (a java.lang.Object)
	at org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
	at java.util.TimerThread.mainLoop(Timer.java:512)
	at java.util.TimerThread.run(Timer.java:462)

Found 1 deadlock.
{quote}




--
This message was sent by Atlassian JIRA
(v6.3.1#6329)