6:41 a.m.
Hi,
We are facing the following situation in our jboss server:
1. Client are using temporary queues for registrations.
2. Clients are sometimes disconnecting without closing the connections.
As result:
1. We have huge amount of waiting thread in the Thread Group: "JBM-threads"
(About 1200)
For example from the jmx-console:
| Thread: jbm-server-session-x2-0q6bpggf-1-xfs9pggf-uuwe9e-c5k1k1a : priority:5,
demon:true, threadId:75, threadState:WAITING,
| lockName:java.lang.Object@11ad3b3
| java.lang.Object.wait(Native Method)
| java.lang.Object.wait(Object.java:474)
| EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
| EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:83)
| java.lang.Thread.run(Thread.java:595)
|
(We have also jbm-client-session threads)
2. We have many of those messages in the log (I know this is our fault - trying to send
messages, but I think it can help illuminating the problem)
| May 21 2008 14:05:51.963 IDT org.jboss.remoting.ServerInvoker ERROR Error executing
server oneway invocation request: Invocat
| ionRequest[13d4aff, JMS, org.jboss.jms.wireformat.SessionSendRequest@106d2f6]
| javax.jms.JMSException: Failed to route Reference[19847052524901780]:NON-RELIABLE to
l1-vq5bqggf-1-z7eaqggf-htbpnh-100a
| at
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:757)
| at
org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:391)
| at
org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop
| (SessionAdvised.java:87)
| at
org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_72806
| 80627620114891.java)
| at
org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
| at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source)
| at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:585)
| at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
| at
org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_72806
| 80627620114891.java)
| at
org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
| at
org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
| at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
| at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:809)
| at org.jboss.remoting.ServerInvoker$1.run(ServerInvoker.java:1815)
| at
org.jboss.jms.server.remoting.DirectThreadPool.run(DirectThreadPool.java:63)
| at
org.jboss.remoting.ServerInvoker.handleOnewayInvocation(ServerInvoker.java:1826)
| at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:758)
| at
org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
| at org.jboss.remoting.Client.invoke(Client.java:1634)
| at org.jboss.remoting.Client.invoke(Client.java:548)
| at org.jboss.remoting.Client.invokeOneway(Client.java:598)
| at org.jboss.remoting.Client.invokeOneway(Client.java:786)
| at org.jboss.remoting.Client.invokeOneway(Client.java:776)
| at
org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:180)
| at
org.jboss.jms.client.delegate.DelegateSupport.doInvokeOneway(DelegateSupport.java:165)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(C
| lientSessionDelegate.java:473)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
| _6145266547759487588.java)
| at
org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)
| at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect28.invoke(SessionAspect28.java)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
| _6145266547759487588.java)
| at
org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
| at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
| _6145266547759487588.java)
| at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send
| _6145266547759487588.java)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)
| at
org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:269)
| at
org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect46.invoke(ProducerAspect46.java)
| at
org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(ClientProducerDelegate$se
| nd_3961598017717988886.java)
| at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
| at
org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(ClientProducerDelegate$se
| nd_3961598017717988886.java)
| at
org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)
| at
org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:164)
| at
org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:207)
| at
org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:145)
| at
org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:136)
| at
com.cisco.nm.dmm.ejb.DMMJ2EEQueueNtfsTask.publishMessage(DMMJ2EEQueueNtfsTask.java:134)
| at
com.cisco.nm.dmm.common.services.QueueTask.publishSynchronizedPageGroups(QueueTask.java:141)
| at
com.cisco.nm.dmm.mdb.SynchronizedPageGroupsMDB.onMessage(SynchronizedPageGroupsMDB.java:117)
| at sun.reflect.GeneratedMethodAccessor523.invoke(Unknown Source)
| at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:585)
| at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)
| at
org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
| at
org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
| at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
| at
org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.j
| ava:54)
| at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
| at
org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
| at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
| at org.jboss.ejb3.tx.BMTInterceptor.handleStateless(BMTInterceptor.java:71)
| at org.jboss.ejb3.tx.BMTInterceptor.invoke(BMTInterceptor.java:131)
| at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
| at
org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
| at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
| at
org.jboss.ejb3.mdb.MessagingContainer.localInvoke(MessagingContainer.java:245)
| at
org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.delivery(MessageInflowLocalProxy.java:268)
| at
org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:138)
| at $Proxy170.onMessage(Unknown Source)
| at
org.jboss.resource.adapter.jms.inflow.JmsServerSession.onMessage(JmsServerSession.java:178)
| at
org.jboss.jms.client.container.ClientConsumer.callOnMessage(ClientConsumer.java:160)
| at
org.jboss.jms.client.container.SessionAspect.handleRun(SessionAspect.java:831)
| at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect14.invoke(SessionAspect14.java)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_
| N8003352271541955702.java)
| at
org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_
| N8003352271541955702.java)
| at
org.jboss.jms.client.delegate.ClientSessionDelegate.run(ClientSessionDelegate.java)
| at org.jboss.jms.client.JBossSession.run(JBossSession.java:199)
| at
org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:237)
| at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:204)
| at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:275)
| at
EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
| at java.lang.Thread.run(Thread.java:595)
|
My questions:
1. What are all those threads? Why we have so many?
2. Can we use any kind of purging mechanism for lost sessions thus reducing thread
numbers?
We are using jboss AS 4.2.0.GA with:
JBM 1.4.0.SP3.CP02
JBR 2.2.2.SP7
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4152312#...
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...