[JBoss Messaging] - Deadlock when receiving messages via https connector - jboss-user

Thursday, 25 September 2008

Topic subscriptions that use https connection factory can result in deadlocks when
BlockingCallbackStore is used.

See stack traces below. But here is a summary of what appears to be happening...

On the client
Timer-3 thread: 
 - Is polling for messages

On the application server
http-0.0.0.0-4459-5 thread (the server-side continuation of Client
Timer-3):
 - Is waiting to lock the ArrayList of callbacks held by
the eventListenerContainer-10 thread.

eventListenerContainer-10 thread:
 - Because the ServerInvokerCallbackHandler has decided that it needs to
persist the callback and is using a BlockingCallbackStore,
eventListenerContainer-10 is also waiting on an object that will be release
when someone calls BlockingCallbackStore getNext() (i.e. its waiting for
http-0.0.0.0-4459-5)

Hence deadlock.

This also results in other appication server threads locking up when they attempt to
commit messages posted to the destination.

We are using JBM 1.4.0.SP3 with JBR 2.2.2.SP4.

(Not sure if its a JB remoting or a JB messaging issue but it manifests in messaging so
I've posted it here for now.)

Do you have any workaround suggestions? (Thought about using the non-blocking
CallbackStore for the connector maybe but the remoting-https-service.xml says not to
change these attributes.)

Client stack (for Timer-3 only):
"Timer-3" daemon prio=6 tid=0x2392d800 nid=0x1050 runnable
[0x2677e000..0x2677fb94]
  |    java.lang.Thread.State: RUNNABLE
  | 	at java.net.SocketInputStream.socketRead0(Native Method)
  | 	at java.net.SocketInputStream.read(Unknown Source)
  | 	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(Unknown Source)
  | 	at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source)
  | 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
  | 	- locked <0x0500b5f0> (a java.lang.Object)
  | 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
  | 	at com.sun.net.ssl.internal.ssl.AppInputStream.read(Unknown Source)
  | 	- locked <0x0500b6c8> (a com.sun.net.ssl.internal.ssl.AppInputStream)
  | 	at java.io.BufferedInputStream.fill(Unknown Source)
  | 	at java.io.BufferedInputStream.read1(Unknown Source)
  | 	at java.io.BufferedInputStream.read(Unknown Source)
  | 	- locked <0x05023b88> (a java.io.BufferedInputStream)
  | 	at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
  | 	at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
  | 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
  | 	- locked <0x05023bc0> (a
sun.net.www.protocol.https.DelegateHttpsURLConnection)
  | 	at java.net.HttpURLConnection.getResponseCode(Unknown Source)
  | 	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
  | 	at
org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:281)
  | 	at
org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:136)
  | 	at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
  | 	at org.jboss.remoting.Client.invoke(Client.java:1634)
  | 	at org.jboss.remoting.Client.invoke(Client.java:548)
  | 	at org.jboss.remoting.Client.getCallbacks(Client.java:1168)
  | 	at org.jboss.remoting.callback.CallbackPoller.run(CallbackPoller.java:188)
  | 	- locked <0x046304f0> (a org.jboss.remoting.callback.CallbackPoller)
  | 	at java.util.TimerThread.mainLoop(Unknown Source)
  | 	at java.util.TimerThread.run(Unknown Source)
  | 

Application server stack (uninteresting threads elided for clarity - can send full stack
if reqd):

2008-09-25 12:09:44
  | Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_03-b05 mixed mode):
  | 
  | "Attach Listener" daemon prio=10 tid=0x0000002af06fcc00 nid=0x36bf waiting
on condition [0x0000000000000000..0x0000000000000000]
  |    java.lang.Thread.State: RUNNABLE
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | "http-0.0.0.0-4459-5" daemon prio=10 tid=0x0000002af2a9c400 nid=0x2bca
waiting for monitor entry [0x000000004cff7000..0x000000004cff7a30]
  |    java.lang.Thread.State: BLOCKED (on object monitor)
  | 	at
org.jboss.remoting.callback.ServerInvokerCallbackHandler.getCallbacks(ServerInvokerCallbackHandler.java:600)
  | 	- waiting to lock <0x0000002ac1765dd0> (a java.util.ArrayList)
  | 	at
org.jboss.remoting.ServerInvoker.handleInternalInvocation(ServerInvoker.java:1513)
  | 	at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:782)
  | 	at org.jboss.remoting.transport.coyote.CoyoteInvoker.service(CoyoteInvoker.java:310)
  | 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
  | 	at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
  | 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | "http-0.0.0.0-4459-3" daemon prio=10 tid=0x0000002aee182400 nid=0x2bc7
runnable [0x000000004b8e0000..0x000000004b8e0bb0]
  |    java.lang.Thread.State: RUNNABLE
  | 	at java.net.SocketInputStream.socketRead0(Native Method)
  | 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
  | 	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
  | 	at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
  | 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)
  | 	- locked <0x0000002aa3a1e688> (a java.lang.Object)
  | 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746)
  | 	at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
  | 	- locked <0x0000002aa3a1e740> (a com.sun.net.ssl.internal.ssl.AppInputStream)
  | 	at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:700)
  | 	at
org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366)
  | 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:805)
  | 	at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
  | 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | "http-0.0.0.0-4459-1" daemon prio=10 tid=0x0000002ae8ef5c00 nid=0x2bb7
runnable [0x0000000050a31000..0x0000000050a31c30]
  |    java.lang.Thread.State: RUNNABLE
  | 	at java.net.SocketInputStream.socketRead0(Native Method)
  | 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
  | 	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
  | 	at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
  | 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)
  | 	- locked <0x0000002aa787c578> (a java.lang.Object)
  | 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746)
  | 	at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
  | 	- locked <0x0000002aa78848c0> (a com.sun.net.ssl.internal.ssl.AppInputStream)
  | 	at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:700)
  | 	at
org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366)
  | 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:805)
  | 	at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
  | 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | 
  | "http-0.0.0.0-443-15" daemon prio=10 tid=0x0000002ae6ea7800 nid=0x2703
waiting for monitor entry [0x000000004f316000..0x000000004f31aa30]
  |    java.lang.Thread.State: BLOCKED (on object monitor)
  | 	at
org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1299)
  | 	- waiting to lock <0x0000002ac174ba68> (a
org.jboss.jms.server.endpoint.ServerSessionEndpoint)
  | 	at
org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:322)
  | 	- locked <0x0000002ac1810058> (a java.lang.Object)
  | 	at
org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
  | 	at
org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:582)
  | 	at
org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:606)
  | 	at
org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:505)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:935)
  | 	- locked <0x0000002ac1764c68> (a java.lang.Object)
  | 	at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
  | 	- locked <0x0000002ad2852870> (a org.jboss.messaging.core.impl.tx.Transaction)
  | 	at
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:474)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionAdvised.java:101)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
  | 	at sun.reflect.GeneratedMethodAccessor463.invoke(Unknown Source)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:119)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
  | 	at
org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
  | 	at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
  | 	at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:795)
  | 	at
org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
  | 	at org.jboss.remoting.Client.invoke(Client.java:1634)
  | 	at org.jboss.remoting.Client.invoke(Client.java:548)
  | 	at org.jboss.remoting.Client.invoke(Client.java:536)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:221)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
  | 	at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:637)
  | 	at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:370)
  | 	at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:238)
  | 	at
org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
  | 	at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:636)
  | 	at
com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2619)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1779)
  | 	- locked <0x0000002ad2852ba8> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
  | 	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
  | 	at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
  | 	at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
  | 	at
org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
  | 	at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
  | 	at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  | 	at $Proxy86.startServer(Unknown Source)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at com.cyclone.services.AdaptorProxyFactoryBean$_A.invoke(np:197)
  | 	at $Proxy112.startServer(Unknown Source)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
  | 	at $Proxy113.startServer(Unknown Source)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at
org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:205)
  | 	at
org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:38)
  | 	at
org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78)
  | 	at
com.cyclone.services.cluster.ClusteredHttpInvokerServiceExporter.invokeAndCreateResult(np:50)
  | 	at
org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:74)
  | 	at
org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
  | 	at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
  | 	at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
  | 	at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
  | 	at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
  | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
  | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  | 	at
com.cyclone.netPrefect.webservices.initialization.ApplicationRunningFilter.doFilter(ApplicationRunningFilter.java:80)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  | 	at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  | 	at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
  | 	at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
  | 	at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
  | 	at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:524)
  | 	at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
  | 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  | 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  | 	at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
  | 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  | 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
  | 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
  | 	at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
  | 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | "http-0.0.0.0-443-14" daemon prio=10 tid=0x0000002ae9746400 nid=0x2702
waiting for monitor entry [0x000000004f214000..0x000000004f219ab0]
  |    java.lang.Thread.State: BLOCKED (on object monitor)
  | 	at
org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1299)
  | 	- waiting to lock <0x0000002ac174ba68> (a
org.jboss.jms.server.endpoint.ServerSessionEndpoint)
  | 	at
org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:322)
  | 	- locked <0x0000002ac1942870> (a java.lang.Object)
  | 	at
org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
  | 	at
org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:582)
  | 	at
org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:606)
  | 	at
org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:505)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:935)
  | 	- locked <0x0000002ac17650a8> (a java.lang.Object)
  | 	at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
  | 	- locked <0x0000002ad2853440> (a org.jboss.messaging.core.impl.tx.Transaction)
  | 	at
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:490)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionAdvised.java:101)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
  | 	at sun.reflect.GeneratedMethodAccessor463.invoke(Unknown Source)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:119)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
  | 	at
org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
  | 	at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
  | 	at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:795)
  | 	at
org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
  | 	at org.jboss.remoting.Client.invoke(Client.java:1634)
  | 	at org.jboss.remoting.Client.invoke(Client.java:548)
  | 	at org.jboss.remoting.Client.invoke(Client.java:536)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:221)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
  | 	at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:637)
  | 	at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:393)
  | 	at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:238)
  | 	at
org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
  | 	at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:487)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3107)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3022)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:2126)
  | 	- locked <0x0000002ad2853778> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1819)
  | 	- locked <0x0000002ad2853778> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
  | 	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
  | 	at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
  | 	at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
  | 	at
org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
  | 	at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
  | 	at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  | 	at $Proxy104.createOrUpdateAutoDiscoveredTransport(Unknown Source)
  | 	at sun.reflect.GeneratedMethodAccessor886.invoke(Unknown Source)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
  | 	at
org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:126)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  | 	at
org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:50)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  | 	at $Proxy105.createOrUpdateAutoDiscoveredTransport(Unknown Source)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at com.cyclone.services.AdaptorProxyFactoryBean$_A.invoke(np:197)
  | 	at $Proxy110.createOrUpdateAutoDiscoveredTransport(Unknown Source)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
  | 	at $Proxy111.createOrUpdateAutoDiscoveredTransport(Unknown Source)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at
org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:205)
  | 	at
org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:38)
  | 	at
org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78)
  | 	at
com.cyclone.services.cluster.ClusteredHttpInvokerServiceExporter.invokeAndCreateResult(np:50)
  | 	at
org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:74)
  | 	at
org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
  | 	at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
  | 	at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
  | 	at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
  | 	at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
  | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
  | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  | 	at
com.cyclone.netPrefect.webservices.initialization.ApplicationRunningFilter.doFilter(ApplicationRunningFilter.java:80)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  | 	at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  | 	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  | 	at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
  | 	at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
  | 	at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
  | 	at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:524)
  | 	at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
  | 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  | 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  | 	at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
  | 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  | 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
  | 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
  | 	at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
  | 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | "eventListenerContainer-10" prio=10 tid=0x0000002ae697f800 nid=0x268c in
Object.wait() [0x000000004b3d9000..0x000000004b3dbab0]
  |    java.lang.Thread.State: WAITING (on object monitor)
  | 	at java.lang.Object.wait(Native Method)
  | 	- waiting on <0x0000002ac17cadc8> (a java.lang.Object)
  | 	at java.lang.Object.wait(Object.java:485)
  | 	at
org.jboss.remoting.callback.BlockingCallbackStore.add(BlockingCallbackStore.java:102)
  | 	- locked <0x0000002ac17cadc8> (a java.lang.Object)
  | 	at
org.jboss.remoting.callback.ServerInvokerCallbackHandler.persistCallback(ServerInvokerCallbackHandler.java:863)
  | 	at
org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:751)
  | 	- locked <0x0000002ac1765dd0> (a java.util.ArrayList)
  | 	at
org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:697)
  | 	at
org.jboss.jms.server.endpoint.ServerSessionEndpoint.performDelivery(ServerSessionEndpoint.java:1432)
  | 	at
org.jboss.jms.server.endpoint.ServerSessionEndpoint.handleDelivery(ServerSessionEndpoint.java:1344)
  | 	- locked <0x0000002ac174ba68> (a
org.jboss.jms.server.endpoint.ServerSessionEndpoint)
  | 	at
org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:322)
  | 	- locked <0x0000002ac180fe30> (a java.lang.Object)
  | 	at
org.jboss.messaging.core.impl.RoundRobinDistributor.handle(RoundRobinDistributor.java:119)
  | 	at
org.jboss.messaging.core.impl.MessagingQueue$DistributorWrapper.handle(MessagingQueue.java:582)
  | 	at
org.jboss.messaging.core.impl.ClusterRoundRobinDistributor.handle(ClusterRoundRobinDistributor.java:79)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport.deliverInternal(ChannelSupport.java:606)
  | 	at
org.jboss.messaging.core.impl.MessagingQueue.deliverInternal(MessagingQueue.java:505)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:935)
  | 	- locked <0x0000002ac1764a48> (a java.lang.Object)
  | 	at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
  | 	- locked <0x0000002ac9974110> (a org.jboss.messaging.core.impl.tx.Transaction)
  | 	at
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:490)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionAdvised.java:101)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
  | 	at sun.reflect.GeneratedMethodAccessor463.invoke(Unknown Source)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:119)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
  | 	at
org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
  | 	at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
  | 	at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:795)
  | 	at
org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
  | 	at org.jboss.remoting.Client.invoke(Client.java:1634)
  | 	at org.jboss.remoting.Client.invoke(Client.java:548)
  | 	at org.jboss.remoting.Client.invoke(Client.java:536)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:221)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
  | 	at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:637)
  | 	at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:393)
  | 	at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:238)
  | 	at
org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
  | 	at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:487)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3107)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3022)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:2126)
  | 	- locked <0x0000002ac9974448> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1819)
  | 	- locked <0x0000002ac9974448> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
  | 	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
  | 	at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
  | 	at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
  | 	at
org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
  | 	at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
  | 	at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  | 	at $Proxy100.onMessage(Unknown Source)
  | 	at
org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
  | 	at
org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
  | 	at
org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
  | 	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
  | 	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
  | 	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
  | 	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 
  | Lots more threads like this...
  | "eventListenerContainer-9" prio=10 tid=0x0000002ae6604000 nid=0x2689 waiting
for monitor entry [0x000000004b0d6000..0x000000004b0d8c30]
  |    java.lang.Thread.State: BLOCKED (on object monitor)
  | 	at
org.jboss.messaging.core.impl.ChannelSupport$InMemoryCallback.afterCommit(ChannelSupport.java:901)
  | 	- waiting to lock <0x0000002ac1764a48> (a java.lang.Object)
  | 	at org.jboss.messaging.core.impl.tx.Transaction.commit(Transaction.java:228)
  | 	- locked <0x0000002ac9d1eaf0> (a org.jboss.messaging.core.impl.tx.Transaction)
  | 	at
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:490)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionAdvised.java:101)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
  | 	at sun.reflect.GeneratedMethodAccessor463.invoke(Unknown Source)
  | 	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 	at java.lang.reflect.Method.invoke(Method.java:597)
  | 	at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:119)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
  | 	at
org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
  | 	at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
  | 	at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:795)
  | 	at
org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
  | 	at org.jboss.remoting.Client.invoke(Client.java:1634)
  | 	at org.jboss.remoting.Client.invoke(Client.java:548)
  | 	at org.jboss.remoting.Client.invoke(Client.java:536)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
  | 	at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:221)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  | 	at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:106)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
  | 	at
org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
  | 	at org.jboss.jms.tx.ResourceManager.sendTransactionXA(ResourceManager.java:637)
  | 	at org.jboss.jms.tx.ResourceManager.commit(ResourceManager.java:393)
  | 	at org.jboss.jms.tx.MessagingXAResource.commit(MessagingXAResource.java:238)
  | 	at
org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.commit(JcaXAResourceWrapper.java:53)
  | 	at
com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:487)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3107)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:3022)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:2126)
  | 	- locked <0x0000002ac99746c0> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1819)
  | 	- locked <0x0000002ac99746c0> (a
com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
  | 	at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
  | 	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
  | 	at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
  | 	at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
  | 	at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
  | 	at
org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)
  | 	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
  | 	at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
  | 	at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
  | 	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  | 	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  | 	at $Proxy100.onMessage(Unknown Source)
  | 	at
org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
  | 	at
org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
  | 	at
org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
  | 	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
  | 	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
  | 	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
  | 	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
  | 	at java.lang.Thread.run(Thread.java:619)
  | 
  |    Locked ownable synchronizers:
  | 	- None
  | 

View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4178801#...

Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[JBoss Messaging] - Deadlock when receiving messages via https connector