[jboss-user] [JBoss Messaging] - Re: JBoss Messaging Locked Up

rtm333 do-not-reply at jboss.com
Tue Jun 5 10:58:57 EDT 2007 

Hi Tim,

The related JIRA entry JBMESSAGING-962 has been marked as non-reproducible. I have to grant that it was very difficult to produce the deadlock. I think I saw it only a couple of times, although the test program ran for more than 24 hours. 

I have now started my test program (adapted to use the new client libs) against JBM 1.3.0.GA on AS 4.2.0.GA. After about 15 minutes on the first run the server locked up (although it did not yet on later runs). These are the server stack frames that seem most relevant to me. The complete server stack trace together with some client stack traces is nearly 2000 lines long. I will happily provide it upon request. 

Thanks,
Ralf Torsten




  | "Thread-892" prio=10 tid=0x01913678 nid=0x3d8 runnable [0xcc7ff000..0xcc7ff8f0]
  |         at java.net.SocketOutputStream.socketWrite0(Native Method)
  |         at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
  |         at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
  |         at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
  |         at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
  |         - locked <0xdac9c868> (a java.io.BufferedOutputStream)
  |         at java.io.DataOutputStream.flush(DataOutputStream.java:106)
  |         at org.jboss.jms.wireformat.ClientDelivery.write(ClientDelivery.java:93)
  |         at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
  |         at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
  |         at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
  |         at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
  |         at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
  |         at org.jboss.remoting.Client.invoke(Client.java:1550)
  |         at org.jboss.remoting.Client.invoke(Client.java:530)
  |         at org.jboss.remoting.Client.invokeOneway(Client.java:580)
  |         at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:687)
  |         at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:564)
  |         at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:296)
  |         - locked <0xdac691c0> (a org.jboss.remoting.transport.bisocket.BisocketClientInvoker)
  |         - locked <0xdac7be88> (a java.lang.Object)
  |         at org.jboss.messaging.core.local.RoundRobinPointToPointRouter.handle(RoundRobinPointToPointRouter.java:120)
  |         at org.jboss.messaging.core.ChannelSupport.deliverInternal(ChannelSupport.java:608)
  |         at org.jboss.messaging.core.ChannelSupport.deliver(ChannelSupport.java:306)
  |         - locked <0xdac66818> (a java.lang.Object)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint$2.run(ServerSessionEndpoint.java:965)
  |         at EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:89)
  |         at java.lang.Thread.run(Thread.java:595)
  | 
  | 
  | "WorkerThread#15[10.196.106.28:60231]" prio=10 tid=0x004ca588 nid=0xb6 waiting for monitor entry [0xcccff000..0xcccffbf0]
  |         at org.jboss.messaging.core.ChannelSupport.handleInternal(ChannelSupport.java:765)
  |         - waiting to lock <0xdac66818> (a java.lang.Object)
  |         at org.jboss.messaging.core.ChannelSupport.handle(ChannelSupport.java:157)
  |         at org.jboss.messaging.core.local.PagingFilteredQueue.handle(PagingFilteredQueue.java:114)
  |         at org.jboss.messaging.core.plugin.postoffice.DefaultPostOffice.route(DefaultPostOffice.java:333)
  |         at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:699)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:297)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:80)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_6145266547759487588.invokeNext(SessionAdvised$send_6145266547759487588.java)
  |         at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:152)
  |         at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
  |         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  |         at java.lang.reflect.Method.invoke(Method.java:585)
  |         at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
  | 
  | "Thread-892" prio=10 tid=0x01913678 nid=0x3d8 runnable [0xcc7ff000..0xcc7ff8f0]
  |         at java.net.SocketOutputStream.socketWrite0(Native Method)
  |         at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
  |         at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
  |         at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
  |         at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
  |         - locked <0xdac9c868> (a java.io.BufferedOutputStream)
  |         at java.io.DataOutputStream.flush(DataOutputStream.java:106)
  |         at org.jboss.jms.wireformat.ClientDelivery.write(ClientDelivery.java:93)
  |         at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
  |         at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
  |         at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
  |         at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
  |         at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
  |         at org.jboss.remoting.Client.invoke(Client.java:1550)
  |         at org.jboss.remoting.Client.invoke(Client.java:530)
  |         at org.jboss.remoting.Client.invokeOneway(Client.java:580)
  |         at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(ServerInvokerCallbackHandler.java:687)
  |         at org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallbackOneway(ServerInvokerCallbackHandler.java:564)
  |         at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.handle(ServerConsumerEndpoint.java:296)
  |         - locked <0xdac691c0> (a org.jboss.remoting.transport.bisocket.BisocketClientInvoker)
  |         - locked <0xdac7be88> (a java.lang.Object)
  |         at org.jboss.messaging.core.local.RoundRobinPointToPointRouter.handle(RoundRobinPointToPointRouter.java:120)
  |         at org.jboss.messaging.core.ChannelSupport.deliverInternal(ChannelSupport.java:608)
  |         at org.jboss.messaging.core.ChannelSupport.deliver(ChannelSupport.java:306)
  |         - locked <0xdac66818> (a java.lang.Object)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint$2.run(ServerSessionEndpoint.java:965)
  |         at EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:89)
  |         at java.lang.Thread.run(Thread.java:595)
  | 
  | 
  | "WorkerThread#15[10.196.106.28:60231]" prio=10 tid=0x004ca588 nid=0xb6 waiting for monitor entry [0xcccff000..0xcccffbf0]
  |         at org.jboss.messaging.core.ChannelSupport.handleInternal(ChannelSupport.java:765)
  |         - waiting to lock <0xdac66818> (a java.lang.Object)
  |         at org.jboss.messaging.core.ChannelSupport.handle(ChannelSupport.java:157)
  |         at org.jboss.messaging.core.local.PagingFilteredQueue.handle(PagingFilteredQueue.java:114)
  |         at org.jboss.messaging.core.plugin.postoffice.DefaultPostOffice.route(DefaultPostOffice.java:333)
  |         at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:699)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:297)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:80)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_6145266547759487588.invokeNext(SessionAdvised$send_6145266547759487588.java)
  |         at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:152)
  |         at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
  |         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  |         at java.lang.reflect.Method.invoke(Method.java:585)
  |         at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_6145266547759487588.invokeNext(SessionAdvised$send_6145266547759487588.java)
  |         at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_6145266547759487588.invokeNext(SessionAdvised$send_6145266547759487588.java)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
  |         at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:85)
  |         at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:141)
  |         - locked <0xd5302b38> (a java.lang.Class)
  |         at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
  |         at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:560)
  |         at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:383)
  |         at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)
  | 
  | 
  | "WorkerThread#18[10.196.32.30:4358]" prio=10 tid=0x006623b8 nid=0x2f4 waiting for monitor entry [0xcc8ff000..0xcc8ffaf0]
  |         at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:105)
  |         - waiting to lock <0xd5302b38> (a java.lang.Class)
  |         at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
  |         at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:560)
  |         at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:383)
  |         at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)
  | 
  | 
  | "Timer-3" daemon prio=10 tid=0x015f4090 nid=0x45 waiting for monitor entry [0xce5ff000..0xce5ffc70]
  |         at org.jboss.messaging.core.ChannelSupport.remove(ChannelSupport.java:214)
  |         - waiting to lock <0xdac66818> (a java.lang.Object)
  |         at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.localClose(ServerConsumerEndpoint.java:473)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint.localClose(ServerSessionEndpoint.java:795)
  |         at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.close(ServerConnectionEndpoint.java:376)
  |         at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleClientFailure(SimpleConnectionManager.java:192)
  |         - locked <0xd9935d08> (a org.jboss.jms.server.connectionmanager.SimpleConnectionManager)
  |         at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleConnectionException(SimpleConnectionManager.java:227)
  |         at org.jboss.remoting.ConnectionNotifier.connectionLost(ConnectionNotifier.java:45)
  |         at org.jboss.remoting.Lease.notifyClientLost(Lease.java:211)
  |         at org.jboss.remoting.Lease.access$300(Lease.java:39)
  |         at org.jboss.remoting.Lease$LeaseTimerTask.run(Lease.java:248)
  |         at java.util.TimerThread.mainLoop(Timer.java:512)
  |         at java.util.TimerThread.run(Timer.java:462)
  | 
  | 
  | 
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_6145266547759487588.invokeNext(SessionAdvised$send_6145266547759487588.java)
  |         at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_6145266547759487588.invokeNext(SessionAdvised$send_6145266547759487588.java)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
  |         at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:85)
  |         at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:141)
  |         - locked <0xd5302b38> (a java.lang.Class)
  |         at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
  |         at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:560)
  |         at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:383)
  |         at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)
  | 
  | 
  | "WorkerThread#18[10.196.32.30:4358]" prio=10 tid=0x006623b8 nid=0x2f4 waiting for monitor entry [0xcc8ff000..0xcc8ffaf0]
  |         at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:105)
  |         - waiting to lock <0xd5302b38> (a java.lang.Class)
  |         at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
  |         at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:560)
  |         at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:383)
  |         at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)
  | 
  | 
  | "Timer-3" daemon prio=10 tid=0x015f4090 nid=0x45 waiting for monitor entry [0xce5ff000..0xce5ffc70]
  |         at org.jboss.messaging.core.ChannelSupport.remove(ChannelSupport.java:214)
  |         - waiting to lock <0xdac66818> (a java.lang.Object)
  |         at org.jboss.jms.server.endpoint.ServerConsumerEndpoint.localClose(ServerConsumerEndpoint.java:473)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint.localClose(ServerSessionEndpoint.java:795)
  |         at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.close(ServerConnectionEndpoint.java:376)
  |         at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleClientFailure(SimpleConnectionManager.java:192)
  |         - locked <0xd9935d08> (a org.jboss.jms.server.connectionmanager.SimpleConnectionManager)
  |         at org.jboss.jms.server.connectionmanager.SimpleConnectionManager.handleConnectionException(SimpleConnectionManager.java:227)
  |         at org.jboss.remoting.ConnectionNotifier.connectionLost(ConnectionNotifier.java:45)
  |         at org.jboss.remoting.Lease.notifyClientLost(Lease.java:211)
  |         at org.jboss.remoting.Lease.access$300(Lease.java:39)
  |         at org.jboss.remoting.Lease$LeaseTimerTask.run(Lease.java:248)
  |         at java.util.TimerThread.mainLoop(Timer.java:512)
  |         at java.util.TimerThread.run(Timer.java:462)
  | 



View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4051404#4051404

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4051404

More information about the jboss-user mailing list