[jboss-user] [JBoss Messaging] - Messages are not getting delivered :(

Tuesday, 29 July 2008

After sending many thousand of messages using JBM 1.4.0.SP3 in a clustered environment,
the messages seem to get stuck. 

What's weird is that it does not get unstuck either without  a server restart - or a
total cluster restart.

I have no clue why.

Here's my thread dump from the JMX console, perhaps there's something in there
that might be revealing:

  | Total Threads: 291
  | Total Thread Groups: 10
  | Timestamp: 20080729090730
  | 
  | Thread Group: system : max priority:10, demon:false
  | 
  |     Thread: Reference Handler : priority:10, demon:true, threadId:2,
threadState:WAITING, lockName:java.lang.ref.Reference$Lock@548123
  | 
  |         java.lang.Object.wait(Native Method)
  |         java.lang.Object.wait(Object.java:474)
  |         java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  | 
  |     Thread: Finalizer : priority:8, demon:true, threadId:3, threadState:WAITING,
lockName:java.lang.ref.ReferenceQueue$Lock@1ea5551
  | 
  |         java.lang.Object.wait(Native Method)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  |         java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
  | 
  |     Thread: Signal Dispatcher : priority:9, demon:true, threadId:4,
threadState:RUNNABLE, lockName:null
  |     Thread: RMI TCP Accept-8508 : priority:5, demon:true, threadId:18,
threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI Reaper : priority:5, demon:false, threadId:20, threadState:WAITING,
lockName:java.lang.ref.ReferenceQueue$Lock@187547e
  | 
  |         java.lang.Object.wait(Native Method)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  |         java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  |         sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:336)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: GC Daemon : priority:2, demon:true, threadId:21,
threadState:TIMED_WAITING, lockName:sun.misc.GC$LatencyLock@1bc3285
  | 
  |         java.lang.Object.wait(Native Method)
  |         sun.misc.GC$Daemon.run(GC.java:100)
  | 
  |     Thread: RMI TCP Accept-8509 : priority:5, demon:true, threadId:28,
threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI TCP Accept-8513 : priority:5, demon:true, threadId:79,
threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI LeaseChecker : priority:5, demon:true, threadId:81,
threadState:TIMED_WAITING, lockName:null
  | 
  |         java.lang.Thread.sleep(Native Method)
  |         sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:310)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  |     Thread: RMI TCP Accept-8514 : priority:5, demon:true, threadId:84,
threadState:RUNNABLE, lockName:null
  | 
  |         java.net.PlainSocketImpl.socketAccept(Native Method)
  |         java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |         java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |         java.net.ServerSocket.accept(ServerSocket.java:421)
  |         sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
  |         java.lang.Thread.run(Thread.java:595)
  | 
  | 
  |     Thread Group: main : max priority:10, demon:false
  | 
  |         Thread: DestroyJavaVM : priority:5, demon:false, threadId:10,
threadState:RUNNABLE, lockName:null
  | 
  |         Thread Group: jboss : max priority:10, demon:false
  | 
  |             Thread: Timer-0 : priority:5, demon:true, threadId:13,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@1713c6f
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Dispatcher-Thread-2 : priority:5, demon:true, threadId:14,
threadState:WAITING, lockName:java.util.ArrayList@136a875
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:545)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Dispatcher-Thread-3 : priority:5, demon:true, threadId:15,
threadState:WAITING, lockName:java.util.ArrayList@16dc715
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:545)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ScannerThread : priority:5, demon:true, threadId:16,
threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                
org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:280)
  |                
org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:225)
  | 
  |             Thread: Timer-1 : priority:5, demon:true, threadId:19,
threadState:WAITING, lockName:java.util.TaskQueue@6df94e
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 java.util.TimerThread.mainLoop(Timer.java:483)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Listener:49220 : priority:5, demon:true, threadId:23,
threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
com.arjuna.ats.internal.arjuna.recovery.Listener.run(Listener.java:119)
  | 
  |             Thread: Thread-6 : priority:5, demon:true, threadId:25,
threadState:TIMED_WAITING,
lockName:com.arjuna.ats.arjuna.coordinator.TransactionReaper@5f7c21
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
com.arjuna.ats.internal.arjuna.coordinator.ReaperThread.run(ReaperThread.java:98)
  | 
  |             Thread: Listener:49236 : priority:5, demon:true, threadId:27,
threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
com.arjuna.ats.internal.arjuna.recovery.Listener.run(Listener.java:119)
  | 
  |             Thread: PooledInvokerAcceptor#0-8510 : priority:5, demon:false,
threadId:29, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.jboss.invocation.pooled.server.PooledInvoker.run(PooledInvoker.java:262)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: AcceptorThread#0:4446 : priority:5, demon:false, threadId:30,
threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.jboss.remoting.transport.socket.SocketServerInvoker.run(SocketServerInvoker.java:520)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ServerSocketRefresh : priority:5, demon:true, threadId:31,
threadState:WAITING, lockName:java.lang.Object@ce62ec
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:704)
  | 
  |             Thread: AcceptorThread#0:8525 : priority:5, demon:false, threadId:32,
threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.jboss.remoting.transport.socket.SocketServerInvoker.run(SocketServerInvoker.java:520)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ServerSocketRefresh : priority:5, demon:true, threadId:33,
threadState:WAITING, lockName:java.lang.Object@1c0abc2
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:704)
  | 
  |             Thread: AcceptorThread#0:8527 : priority:5, demon:false, threadId:34,
threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.jboss.remoting.transport.socket.SocketServerInvoker.run(SocketServerInvoker.java:520)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: secondaryServerSocketThread : priority:5, demon:true, threadId:35,
threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.jboss.remoting.transport.bisocket.BisocketServerInvoker$SecondaryServerSocketThread.run(BisocketServerInvoker.java:873)
  | 
  |             Thread: ServerSocketRefresh : priority:5, demon:true, threadId:36,
threadState:WAITING, lockName:java.lang.Object@4c7fb1
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:704)
  | 
  |             Thread: Timer-2 : priority:5, demon:true, threadId:37,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@53563c
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-3 : priority:5, demon:true, threadId:47,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@142b5ba
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: AsynchKeyChangeHandler Thread : priority:5, demon:false,
threadId:77, threadState:WAITING, lockName:java.lang.Object@a8d765
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
  |                
org.jboss.ha.framework.server.AsynchEventHandler.run(AsynchEventHandler.java:102)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: AsynchViewChangeHandler Thread : priority:5, demon:false,
threadId:78, threadState:WAITING, lockName:java.lang.Object@d539a9
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
  |                
org.jboss.ha.framework.server.AsynchEventHandler.run(AsynchEventHandler.java:102)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: PooledInvokerAcceptor#0-8515 : priority:5, demon:false,
threadId:85, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.jboss.invocation.pooled.server.PooledInvoker.run(PooledInvoker.java:262)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ContainerBackgroundProcessor[StandardEngine[jboss.web]] :
priority:5, demon:true, threadId:86, threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1578)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Timer-4 : priority:5, demon:false, threadId:87,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@bcc53f
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-0 : priority:5, demon:false,
threadId:90, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-1 : priority:5, demon:false,
threadId:91, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-2 : priority:5, demon:false,
threadId:92, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-3 : priority:5, demon:false,
threadId:93, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-4 : priority:5, demon:false,
threadId:94, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-5 : priority:5, demon:false,
threadId:95, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-6 : priority:5, demon:false,
threadId:96, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-7 : priority:5, demon:false,
threadId:97, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-8 : priority:5, demon:false,
threadId:98, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: DefaultQuartzScheduler_Worker-9 : priority:5, demon:false,
threadId:99, threadState:TIMED_WAITING, lockName:java.lang.Object@139ce5a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
  |                
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
  |                
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
  | 
  |             Thread: Timer-5 : priority:5, demon:true, threadId:103,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@1a9a37
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: IdleRemover : priority:5, demon:true, threadId:104,
threadState:TIMED_WAITING, lockName:java.util.ArrayList@82bd0a
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.jboss.resource.connectionmanager.IdleRemover$IdleRemoverRunnable.run(IdleRemover.java:161)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Thread-14 : priority:5, demon:true, threadId:106,
threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                
org.jboss.messaging.core.impl.memory.SimpleMemoryManager$MemoryRunnable.run(SimpleMemoryManager.java:126)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Timer-6 : priority:5, demon:true, threadId:109,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@149132d
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-7 : priority:5, demon:true, threadId:111,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@8a2e96
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Scheduler main thread : priority:5, demon:true, threadId:115,
threadState:WAITING, lockName:java.lang.Object@1f8b8b4
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jgroups.util.Queue.peek(Queue.java:404)
  |                 org.jgroups.util.Scheduler.run(Scheduler.java:77)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: ReusableThread : priority:5, demon:true, threadId:127,
threadState:WAITING, lockName:org.jgroups.util.ReusableThread@26347c
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 org.jgroups.util.ReusableThread.run(ReusableThread.java:201)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Timer-8 : priority:5, demon:true, threadId:141,
threadState:WAITING, lockName:java.util.TaskQueue@3fbf1f
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                 java.util.TimerThread.mainLoop(Timer.java:483)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-9 : priority:5, demon:true, threadId:142,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@11c9cc3
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-10 : priority:5, demon:true, threadId:144,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@177e380
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#0[172.17.6.137:52243] : priority:5, demon:false,
threadId:149, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: Timer-12 : priority:5, demon:true, threadId:150,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@1f8eb4e
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#1[172.17.6.137:52214] : priority:5, demon:false,
threadId:152, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: Timer-13 : priority:5, demon:true, threadId:153,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@1204ee4
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: JCA PoolFiller : priority:5, demon:false, threadId:160,
threadState:WAITING, lockName:java.util.LinkedList@1b9ae9
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.resource.connectionmanager.PoolFiller.run(PoolFiller.java:87)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: WorkerThread#2[172.17.6.137:55072] : priority:5, demon:false,
threadId:164, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#3[172.17.6.136:43630] : priority:5, demon:false,
threadId:165, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#4[172.17.6.137:44955] : priority:5, demon:false,
threadId:166, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#5[172.17.6.137:52241] : priority:5, demon:false,
threadId:170, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#6[172.17.6.137:56737] : priority:5, demon:false,
threadId:171, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#7[172.17.6.137:55073] : priority:5, demon:false,
threadId:172, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#8[172.17.6.137:52190] : priority:5, demon:false,
threadId:173, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#9[172.17.6.137:52242] : priority:5, demon:false,
threadId:174, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: Timer-14 : priority:5, demon:false, threadId:175,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@576222
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#10[172.17.6.136:48915] : priority:5, demon:false,
threadId:176, threadState:WAITING,
lockName:org.jboss.remoting.transport.socket.ServerThread@1145cf3
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:219)
  | 
  |             Thread: Timer-15 : priority:5, demon:false, threadId:177,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@ebd798
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: Timer-16 : priority:5, demon:false, threadId:178,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@19740bb
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: WorkerThread#11[172.17.6.136:43075] : priority:5, demon:false,
threadId:197, threadState:TIMED_WAITING, lockName:java.lang.Object@11acf73
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                
org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                
org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                
org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                
org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                
org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                
org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  | 
  |             Thread: Timer-17 : priority:5, demon:false, threadId:198,
threadState:TIMED_WAITING, lockName:java.util.TaskQueue@1298ec4
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.util.TimerThread.mainLoop(Timer.java:509)
  |                 java.util.TimerThread.run(Timer.java:462)
  | 
  |             Thread: JBossLifeThread : priority:5, demon:false, threadId:199,
threadState:WAITING, lockName:java.lang.Object@15bfaf8
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.system.server.ServerImpl$LifeThread.run(ServerImpl.java:940)
  | 
  |             Thread: http-0.0.0.0-9000-Acceptor-0 : priority:5, demon:true,
threadId:200, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.PlainSocketImpl.socketAccept(Native Method)
  |                 java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  |                 java.net.ServerSocket.implAccept(ServerSocket.java:450)
  |                 java.net.ServerSocket.accept(ServerSocket.java:421)
  |                
org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
  |                
org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:309)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: org.springframework.util.WeakReferenceMonitor : priority:5,
demon:true, threadId:201, threadState:WAITING,
lockName:java.lang.ref.ReferenceQueue$Lock@1b9e4cb
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  |                 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  |                
org.springframework.util.WeakReferenceMonitor$MonitoringProcess.run(WeakReferenceMonitor.java:132)
  |                 java.lang.Thread.run(Thread.java:595)
  | 
  |             Thread: Thread-7 : priority:5, demon:false, threadId:26,
threadState:TIMED_WAITING, lockName:null
  | 
  |                 java.lang.Thread.sleep(Native Method)
  |                
com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWork(PeriodicRecovery.java:248)
  |                
com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:163)
  | 
  |             Thread: WorkerThread#13[172.17.6.136:39892] : priority:5, demon:false,
threadId:208, threadState:TIMED_WAITING, lockName:java.lang.Object@15bf036
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                
org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                
org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                
org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                
org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                
org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                
org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  |                 
  |             Thread: WorkerThread#14[172.17.6.137:51381] : priority:5, demon:false,
threadId:209, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#15[172.17.6.136:43674] : priority:5, demon:false,
threadId:210, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#16[172.17.6.137:32774] : priority:5, demon:false,
threadId:211, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#17[172.17.6.137:52183] : priority:5, demon:false,
threadId:212, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#18[172.17.6.137:44954] : priority:5, demon:false,
threadId:213, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#19[172.17.6.137:60670] : priority:5, demon:false,
threadId:214, threadState:WAITING,
lockName:org.jboss.remoting.transport.socket.ServerThread@1039fc0
  | 
  |                 java.lang.Object.wait(Native Method)
  |                 java.lang.Object.wait(Object.java:474)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:219)
  | 
  |             Thread: WorkerThread#20[172.17.6.136:50276] : priority:5, demon:false,
threadId:215, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#21[172.17.6.136:43631] : priority:5, demon:false,
threadId:217, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#22[172.17.6.136:50962] : priority:5, demon:false,
threadId:219, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#23[172.17.6.136:43629] : priority:5, demon:false,
threadId:220, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#24[172.17.6.136:50939] : priority:5, demon:false,
threadId:221, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#25[172.17.6.136:43658] : priority:5, demon:false,
threadId:222, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#26[172.17.6.136:43632] : priority:5, demon:false,
threadId:223, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#27[172.17.6.136:43633] : priority:5, demon:false,
threadId:224, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#28[172.17.6.137:52244] : priority:5, demon:false,
threadId:225, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#29[172.17.6.136:50277] : priority:5, demon:false,
threadId:228, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#30[172.17.6.136:34646] : priority:5, demon:false,
threadId:230, threadState:TIMED_WAITING, lockName:java.lang.Object@f7a986
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                
org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                
org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                
org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                
org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                
org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                
org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  | 
  |             Thread: WorkerThread#31[172.17.6.136:43625] : priority:5, demon:false,
threadId:233, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#32[172.17.6.136:33657] : priority:5, demon:false,
threadId:244, threadState:TIMED_WAITING, lockName:java.lang.Object@19aac4c
  | 
  |                 java.lang.Object.wait(Native Method)
  |                
org.jboss.jms.client.container.ClientConsumer.waitOnLock(ClientConsumer.java:775)
  |                
org.jboss.jms.client.container.ClientConsumer.getMessage(ClientConsumer.java:822)
  |                
org.jboss.jms.client.container.ClientConsumer.receive(ClientConsumer.java:460)
  |                
org.jboss.jms.client.container.ConsumerAspect.handleReceive(ConsumerAspect.java:191)
  |                
org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect52.invoke(ConsumerAspect52.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |                
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
  |                
org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
  |                
org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
  |                
org.jboss.resource.adapter.jms.JmsMessageConsumer.receive(JmsMessageConsumer.java:112)
  |                 ...
  | 
  |             Thread: WorkerThread#33[172.17.6.136:50275] : priority:5, demon:false,
threadId:253, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#34[172.17.6.137:52197] : priority:5, demon:false,
threadId:268, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#35[172.17.6.137:56331] : priority:5, demon:false,
threadId:274, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#36[172.17.6.136:43626] : priority:5, demon:false,
threadId:281, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#37[172.17.6.136:50266] : priority:5, demon:false,
threadId:282, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#38[172.17.6.136:43624] : priority:5, demon:false,
threadId:311, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#39[172.17.6.136:43628] : priority:5, demon:false,
threadId:312, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#40[172.17.6.137:60066] : priority:5, demon:false,
threadId:324, threadState:RUNNABLE, lockName:null
  | 
  |                 java.net.SocketInputStream.socketRead0(Native Method)
  |                 java.net.SocketInputStream.read(SocketInputStream.java:129)
  |                 java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  |                 java.io.BufferedInputStream.read(BufferedInputStream.java:235)
  |                 java.io.FilterInputStream.read(FilterInputStream.java:66)
  |                
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
  |                
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
  |                
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
  |                
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
  | 
  |             Thread: WorkerThread#41[172.17.6.136:50278] : priority:5, demon:false,
threadId:365

View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4167386#...

Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[jboss-user] [JBoss Messaging] - Messages are not getting delivered :(