[jboss-user] [JBoss Messaging] - Messages are not getting delivered :(

TheOverlord do-not-reply at jboss.com
Tue Jul 29 12:19:45 EDT 2008


After sending many thousand of messages using JBM 1.4.0.SP3 in a clustered environment, the messages seem to get stuck. 

What's weird is that it does not get unstuck either without  a server restart - or a total cluster restart.

I have no clue why.

Here's my thread dump from the JMX console, perhaps there's something in there that might be revealing:


  | Total Threads: 291
  | Total Thread Groups: 10
  | Timestamp: 20080729090730
  | 
  | Thread Group: system : max priority:10, demon:false
  | 
  |     Thread: Reference Handler : priority:10, demon:true, threadId:2, threadState:WAITING, lockName:java.lang.ref.Reference$Lock at 548123
  | 
  |         java.lang.Object.wait(Native Method)
  |         java.lang.Object.wait(Object.java:474)
  |         java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  | 
  |     Thread: Finalizer : priority:8, demon:true, threadId:3, threadState:WAITING, lockName:java.lang.ref.ReferenceQueue$Lock at 1ea5551
  | 
  |         java.lang.Object.wait(Native Method)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  |         java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
  | 
  |     Thread: Signal Dispatcher : priority:9, demon:true, threadId:4, threadState:RUNNABLE, lockName:null
  |     Thread: RMI TCP Accept-8508 : priority:5, demon:true, threadId:18, threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI Reaper : priority:5, demon:false, threadId:20, threadState:WAITING, lockName:java.lang.ref.ReferenceQueue$Lock at 187547e
  | 
  |         java.lang.Object.wait(Native Method)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  |         sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:336)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: GC Daemon : priority:2, demon:true, threadId:21, threadState:TIMED_WAITING, lockName:sun.misc.GC$LatencyLock at 1bc3285
  | 
  |         java.lang.Object.wait(Native Method)
  |         sun.misc.GC$Daemon.run(GC.java:100)
  | 
  |     Thread: RMI TCP Accept-8509 : priority:5, demon:true, threadId:28, threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI TCP Accept-8513 : priority:5, demon:true, threadId:79, threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI LeaseChecker : priority:5, demon:true, threadId:81, threadState:TIMED_WAITING, lockName:null
  | 
  |         java.lang.Thread.sleep(Native Method)
  |         sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:310)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI TCP Accept-8514 : priority:5, demon:true, threadId:84, threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  | 
  |     Thread Group: main : max priority:10, demon:false
  | 
  |         Thread: DestroyJavaVM : priority:5, demon:false, threadId:10, threadState:RUNNABLE, lockName:null
  | 
  |         Thread Group: jboss : max priority:10, demon:false
  | 
  |             Thread: Timer-0 : priority:5, demon:true, threadId:13, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 1713c6f
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Dispatcher-Thread-2 : priority:5, demon:true, threadId:14, threadState:WAITING, lockName:java.util.ArrayList at 136a875
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:545)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Dispatcher-Thread-3 : priority:5, demon:true, threadId:15, threadState:WAITING, lockName:java.util.ArrayList at 16dc715
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:545)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ScannerThread : priority:5, demon:true, threadId:16, threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                 org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:280)
  |                 org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:225)
  | 
  |             Thread: Timer-1 : priority:5, demon:true, threadId:19, threadState:WAITING, lockName:java.util.TaskQueue at 6df94e
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 java.util.TimerThread.mainLoop(Timer.java:483)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Listener:49220 : priority:5, demon:true, threadId:23, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 com.arjuna.ats.internal.arjuna.recovery.Listener.run(Listener.java:119)
  | 
  |             Thread: Thread-6 : priority:5, demon:true, threadId:25, threadState:TIMED_WAITING, lockName:com.arjuna.ats.arjuna.coordinator.TransactionReaper at 5f7c21
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 com.arjuna.ats.internal.arjuna.coordinator.ReaperThread.run(ReaperThread.java:98)
  | 
  |             Thread: Listener:49236 : priority:5, demon:true, threadId:27, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 com.arjuna.ats.internal.arjuna.recovery.Listener.run(Listener.java:119)
  | 
  |             Thread: PooledInvokerAcceptor#0-8510 : priority:5, demon:false, threadId:29, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.jboss.invocation.pooled.server.PooledInvoker.run(PooledInvoker.java:262)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: AcceptorThread#0:4446 : priority:5, demon:false, threadId:30, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.jboss.remoting.transport.socket.SocketServerInvoker.run(SocketServerInvoker.java:520)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ServerSocketRefresh : priority:5, demon:true, threadId:31, threadState:WAITING, lockName:java.lang.Object at ce62ec
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:704)
  | 
  |             Thread: AcceptorThread#0:8525 : priority:5, demon:false, threadId:32, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.jboss.remoting.transport.socket.SocketServerInvoker.run(SocketServerInvoker.java:520)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ServerSocketRefresh : priority:5, demon:true, threadId:33, threadState:WAITING, lockName:java.lang.Object at 1c0abc2
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:704)
  | 
  |             Thread: AcceptorThread#0:8527 : priority:5, demon:false, threadId:34, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.jboss.remoting.transport.socket.SocketServerInvoker.run(SocketServerInvoker.java:520)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: secondaryServerSocketThread : priority:5, demon:true, threadId:35, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.jboss.remoting.transport.bisocket.BisocketServerInvoker$SecondaryServerSocketThread.run(BisocketServerInvoker.java:873)
  | 
  |             Thread: ServerSocketRefresh : priority:5, demon:true, threadId:36, threadState:WAITING, lockName:java.lang.Object at 4c7fb1
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:704)
  | 
  |             Thread: Timer-2 : priority:5, demon:true, threadId:37, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 53563c
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-3 : priority:5, demon:true, threadId:47, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 142b5ba
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: AsynchKeyChangeHandler Thread : priority:5, demon:false, threadId:77, threadState:WAITING, lockName:java.lang.Object at a8d765
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
  |                 org.jboss.ha.framework.server.AsynchEventHandler.run(AsynchEventHandler.java:102)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: AsynchViewChangeHandler Thread : priority:5, demon:false, threadId:78, threadState:WAITING, lockName:java.lang.Object at d539a9
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
  |                 org.jboss.ha.framework.server.AsynchEventHandler.run(AsynchEventHandler.java:102)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: PooledInvokerAcceptor#0-8515 : priority:5, demon:false, threadId:85, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.jboss.invocation.pooled.server.PooledInvoker.run(PooledInvoker.java:262)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ContainerBackgroundProcessor[StandardEngine[jboss.web]] : priority:5, demon:true, threadId:86, threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                 org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1578)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Timer-4 : priority:5, demon:false, threadId:87, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at bcc53f
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-0 : priority:5, demon:false, threadId:90, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-1 : priority:5, demon:false, threadId:91, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-2 : priority:5, demon:false, threadId:92, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-3 : priority:5, demon:false, threadId:93, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-4 : priority:5, demon:false, threadId:94, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-5 : priority:5, demon:false, threadId:95, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-6 : priority:5, demon:false, threadId:96, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-7 : priority:5, demon:false, threadId:97, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-8 : priority:5, demon:false, threadId:98, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-9 : priority:5, demon:false, threadId:99, threadState:TIMED_WAITING, lockName:java.lang.Object at 139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                 org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: Timer-5 : priority:5, demon:true, threadId:103, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 1a9a37
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: IdleRemover : priority:5, demon:true, threadId:104, threadState:TIMED_WAITING, lockName:java.util.ArrayList at 82bd0a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.jboss.resource.connectionmanager.IdleRemover$IdleRemoverRunnable.run(IdleRemover.java:161)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Thread-14 : priority:5, demon:true, threadId:106, threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                 org.jboss.messaging.core.impl.memory.SimpleMemoryManager$MemoryRunnable.run(SimpleMemoryManager.java:126)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Timer-6 : priority:5, demon:true, threadId:109, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 149132d
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-7 : priority:5, demon:true, threadId:111, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 8a2e96
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Scheduler main thread : priority:5, demon:true, threadId:115, threadState:WAITING, lockName:java.lang.Object at 1f8b8b4
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jgroups.util.Queue.peek(Queue.java:404)
  |                 org.jgroups.util.Scheduler.run(Scheduler.java:77)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ReusableThread : priority:5, demon:true, threadId:127, threadState:WAITING, lockName:org.jgroups.util.ReusableThread at 26347c
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jgroups.util.ReusableThread.run(ReusableThread.java:201)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Timer-8 : priority:5, demon:true, threadId:141, threadState:WAITING, lockName:java.util.TaskQueue at 3fbf1f
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 java.util.TimerThread.mainLoop(Timer.java:483)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-9 : priority:5, demon:true, threadId:142, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 11c9cc3
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-10 : priority:5, demon:true, threadId:144, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 177e380
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#0[172.17.6.137:52243] : priority:5, demon:false, threadId:149, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: Timer-12 : priority:5, demon:true, threadId:150, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 1f8eb4e
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#1[172.17.6.137:52214] : priority:5, demon:false, threadId:152, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: Timer-13 : priority:5, demon:true, threadId:153, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 1204ee4
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: JCA PoolFiller : priority:5, demon:false, threadId:160, threadState:WAITING, lockName:java.util.LinkedList at 1b9ae9
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.resource.connectionmanager.PoolFiller.run(PoolFiller.java:87)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: WorkerThread#2[172.17.6.137:55072] : priority:5, demon:false, threadId:164, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#3[172.17.6.136:43630] : priority:5, demon:false, threadId:165, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#4[172.17.6.137:44955] : priority:5, demon:false, threadId:166, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#5[172.17.6.137:52241] : priority:5, demon:false, threadId:170, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#6[172.17.6.137:56737] : priority:5, demon:false, threadId:171, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#7[172.17.6.137:55073] : priority:5, demon:false, threadId:172, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#8[172.17.6.137:52190] : priority:5, demon:false, threadId:173, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#9[172.17.6.137:52242] : priority:5, demon:false, threadId:174, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: Timer-14 : priority:5, demon:false, threadId:175, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 576222
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#10[172.17.6.136:48915] : priority:5, demon:false, threadId:176, threadState:WAITING, lockName:org.jboss.remoting.transport.socket.ServerThread at 1145cf3
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:219)
  | 
  |             Thread: Timer-15 : priority:5, demon:false, threadId:177, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at ebd798
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-16 : priority:5, demon:false, threadId:178, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 19740bb
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#11[172.17.6.136:43075] : priority:5, demon:false, threadId:197, threadState:TIMED_WAITING, lockName:java.lang.Object at 11acf73
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                 org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                 org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                 org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                 org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                 org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                 org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                 org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  | 
  |             Thread: Timer-17 : priority:5, demon:false, threadId:198, threadState:TIMED_WAITING, lockName:java.util.TaskQueue at 1298ec4
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: JBossLifeThread : priority:5, demon:false, threadId:199, threadState:WAITING, lockName:java.lang.Object at 15bfaf8
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.system.server.ServerImpl$LifeThread.run(ServerImpl.java:940)
  | 
  |             Thread: http-0.0.0.0-9000-Acceptor-0 : priority:5, demon:true, threadId:200, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                 org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
  |                 org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:309)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: org.springframework.util.WeakReferenceMonitor : priority:5, demon:true, threadId:201, threadState:WAITING, lockName:java.lang.ref.ReferenceQueue$Lock at 1b9e4cb
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  |                 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  |                 org.springframework.util.WeakReferenceMonitor$MonitoringProcess.run(WeakReferenceMonitor.java:132)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Thread-7 : priority:5, demon:false, threadId:26, threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                 com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWork(PeriodicRecovery.java:248)
  |                 com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:163)
  | 
  |             Thread: WorkerThread#13[172.17.6.136:39892] : priority:5, demon:false, threadId:208, threadState:TIMED_WAITING, lockName:java.lang.Object at 15bf036
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                 org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                 org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                 org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                 org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                 org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                 org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                 org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  |                 
  |             Thread: WorkerThread#14[172.17.6.137:51381] : priority:5, demon:false, threadId:209, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#15[172.17.6.136:43674] : priority:5, demon:false, threadId:210, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#16[172.17.6.137:32774] : priority:5, demon:false, threadId:211, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#17[172.17.6.137:52183] : priority:5, demon:false, threadId:212, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#18[172.17.6.137:44954] : priority:5, demon:false, threadId:213, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#19[172.17.6.137:60670] : priority:5, demon:false, threadId:214, threadState:WAITING, lockName:org.jboss.remoting.transport.socket.ServerThread at 1039fc0
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:219)
  | 
  |             Thread: WorkerThread#20[172.17.6.136:50276] : priority:5, demon:false, threadId:215, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#21[172.17.6.136:43631] : priority:5, demon:false, threadId:217, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#22[172.17.6.136:50962] : priority:5, demon:false, threadId:219, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#23[172.17.6.136:43629] : priority:5, demon:false, threadId:220, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#24[172.17.6.136:50939] : priority:5, demon:false, threadId:221, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#25[172.17.6.136:43658] : priority:5, demon:false, threadId:222, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#26[172.17.6.136:43632] : priority:5, demon:false, threadId:223, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#27[172.17.6.136:43633] : priority:5, demon:false, threadId:224, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#28[172.17.6.137:52244] : priority:5, demon:false, threadId:225, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#29[172.17.6.136:50277] : priority:5, demon:false, threadId:228, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#30[172.17.6.136:34646] : priority:5, demon:false, threadId:230, threadState:TIMED_WAITING, lockName:java.lang.Object at f7a986
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                 org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                 org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                 org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                 org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                 org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                 org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                 org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  | 
  |             Thread: WorkerThread#31[172.17.6.136:43625] : priority:5, demon:false, threadId:233, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#32[172.17.6.136:33657] : priority:5, demon:false, threadId:244, threadState:TIMED_WAITING, lockName:java.lang.Object at 19aac4c
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                 org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                 org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                 org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                 org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                 org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                 org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                 org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                 org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  | 
  |             Thread: WorkerThread#33[172.17.6.136:50275] : priority:5, demon:false, threadId:253, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#34[172.17.6.137:52197] : priority:5, demon:false, threadId:268, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#35[172.17.6.137:56331] : priority:5, demon:false, threadId:274, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#36[172.17.6.136:43626] : priority:5, demon:false, threadId:281, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#37[172.17.6.136:50266] : priority:5, demon:false, threadId:282, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#38[172.17.6.136:43624] : priority:5, demon:false, threadId:311, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#39[172.17.6.136:43628] : priority:5, demon:false, threadId:312, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#40[172.17.6.137:60066] : priority:5, demon:false, threadId:324, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                 org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                 org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                 org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                 org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#41[172.17.6.136:50278] : priority:5, demon:false, threadId:365

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4167386#4167386

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4167386



More information about the jboss-user mailing list