[jboss-user] [JBoss Messaging] - Dealing with lost JMS sessions

ykonrad do-not-reply at jboss.com
Wed May 21 07:41:03 EDT 2008


Hi,

We are facing the following situation in our jboss server:

1. Client are using temporary queues for registrations. 
2. Clients are sometimes disconnecting without closing the connections.

As result:

1. We have huge amount of waiting thread in the Thread Group: "JBM-threads" (About 1200) 
 For example from the jmx-console:


  | Thread: jbm-server-session-x2-0q6bpggf-1-xfs9pggf-uuwe9e-c5k1k1a : priority:5, demon:true, threadId:75, threadState:WAITING, 
  | lockName:java.lang.Object at 11ad3b3
  | 	java.lang.Object.wait(Native Method)
  | 	java.lang.Object.wait(Object.java:474)
  | 	EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
  | 	EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:83)
  | 	java.lang.Thread.run(Thread.java:595)
  | 

(We have also jbm-client-session threads)

2. We have many of those messages in the log (I know this is our fault - trying to send messages, but I think it can help illuminating the problem)


  | May 21 2008 14:05:51.963 IDT org.jboss.remoting.ServerInvoker ERROR Error executing server oneway invocation request: Invocat
  | ionRequest[13d4aff, JMS, org.jboss.jms.wireformat.SessionSendRequest at 106d2f6]
  | javax.jms.JMSException: Failed to route Reference[19847052524901780]:NON-RELIABLE to l1-vq5bqggf-1-z7eaqggf-htbpnh-100a
  |         at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:757)
  |         at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:391)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop
  | (SessionAdvised.java:87)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_72806
  | 80627620114891.java)
  |         at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
  |         at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source)
  |         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  |         at java.lang.reflect.Method.invoke(Method.java:585)
  |         at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_72806
  | 80627620114891.java)
  |         at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
  |         at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
  |         at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
  |         at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:809)
  |         at org.jboss.remoting.ServerInvoker$1.run(ServerInvoker.java:1815)
  |         at org.jboss.jms.server.remoting.DirectThreadPool.run(DirectThreadPool.java:63)
  |         at org.jboss.remoting.ServerInvoker.handleOnewayInvocation(ServerInvoker.java:1826)
  |         at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:758)
  |         at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
  |         at org.jboss.remoting.Client.invoke(Client.java:1634)
  |         at org.jboss.remoting.Client.invoke(Client.java:548)
  |         at org.jboss.remoting.Client.invokeOneway(Client.java:598)
  |         at org.jboss.remoting.Client.invokeOneway(Client.java:786)
  |         at org.jboss.remoting.Client.invokeOneway(Client.java:776)
  |         at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:180)
  |         at org.jboss.jms.client.delegate.DelegateSupport.doInvokeOneway(DelegateSupport.java:165)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(C
  | lientSessionDelegate.java:473)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
  | _6145266547759487588.java)
  |         at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)
  |         at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect28.invoke(SessionAspect28.java)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
  | _6145266547759487588.java)
  |         at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
  |         at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
  | _6145266547759487588.java)
  |         at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |         at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
  | _6145266547759487588.java)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)
  |         at org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:269)
  |         at org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect46.invoke(ProducerAspect46.java)
  |         at org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(ClientProducerDelegate$se
  | nd_3961598017717988886.java)
  |         at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |         at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |         at org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(ClientProducerDelegate$se
  | nd_3961598017717988886.java)
  |         at org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)
  |         at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:164)
  |         at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:207)
  |         at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:145)
  |         at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:136)
  |         at com.cisco.nm.dmm.ejb.DMMJ2EEQueueNtfsTask.publishMessage(DMMJ2EEQueueNtfsTask.java:134)
  |         at com.cisco.nm.dmm.common.services.QueueTask.publishSynchronizedPageGroups(QueueTask.java:141)
  |         at com.cisco.nm.dmm.mdb.SynchronizedPageGroupsMDB.onMessage(SynchronizedPageGroupsMDB.java:117)
  |         at sun.reflect.GeneratedMethodAccessor523.invoke(Unknown Source)
  |         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  |         at java.lang.reflect.Method.invoke(Method.java:585)
  |         at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)
  |         at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
  |         at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
  |         at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
  |         at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.j
  | ava:54)
  |         at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
  |         at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
  |         at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
  |         at org.jboss.ejb3.tx.BMTInterceptor.handleStateless(BMTInterceptor.java:71)
  |         at org.jboss.ejb3.tx.BMTInterceptor.invoke(BMTInterceptor.java:131)
  |         at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
  |         at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
  |         at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
  |         at org.jboss.ejb3.mdb.MessagingContainer.localInvoke(MessagingContainer.java:245)
  |         at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.delivery(MessageInflowLocalProxy.java:268)
  |         at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:138)
  |         at $Proxy170.onMessage(Unknown Source)
  |         at org.jboss.resource.adapter.jms.inflow.JmsServerSession.onMessage(JmsServerSession.java:178)
  |         at org.jboss.jms.client.container.ClientConsumer.callOnMessage(ClientConsumer.java:160)
  |         at org.jboss.jms.client.container.SessionAspect.handleRun(SessionAspect.java:831)
  |         at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect14.invoke(SessionAspect14.java)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_
  | N8003352271541955702.java)
  |         at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
  |         at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_
  | N8003352271541955702.java)
  |         at org.jboss.jms.client.delegate.ClientSessionDelegate.run(ClientSessionDelegate.java)
  |         at org.jboss.jms.client.JBossSession.run(JBossSession.java:199)
  |         at org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:237)
  |         at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:204)
  |         at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:275)
  |         at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
  |         at java.lang.Thread.run(Thread.java:595)
  | 

My questions:
1. What are all those threads? Why we have so many?
2. Can we use any kind of purging mechanism for lost sessions thus reducing thread numbers?

We are using jboss AS 4.2.0.GA with: 
JBM 1.4.0.SP3.CP02
JBR 2.2.2.SP7

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4152312#4152312

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4152312



More information about the jboss-user mailing list