3:29 p.m.
Rajasekhar Devi Reddy [http://community.jboss.org/people/devireddy] created the
discussion
"Issues while posting large number for messages to the Queue"
To view the discussion, visit: http://community.jboss.org/message/562263#562263
--------------------------------------------------------------
+**I am sorry if this had to be posted to Remoting forum. I was not sure so I posted it
here. I have also read other posts that talk abut similar issues but all of the posts
there tell me that these issues should be resolved in the JBM & Remoting versions I am
using (or in some earlier versions). But I still see them.+
Environment/setup
* Solaris SunOS 5.10
* java 1.6.0_20 *64* bit
* JBoss AS 4.2.3
* JBoss Messaging 1.4.5 GA
* Remoting 2.2.3 sp2
* Cluster of two nodes
* Two queues (clustered)
* Two MDBs one for each queue, MDB pool size is set to 30 for each.
* Databsae Oracle 10g
* Number of DefaultDS connections (form JMS persistence) is set to a maximum of 100.
We have an application that uses JMS queues and MDB to achieve asynchronous processing. We
have two queues and two MDBs that receive and processes messages from these queues. Work
flow starts by posting messages on to queue 1, MDB 1 picks up, processes the messages and
write a message on to queue2. MDB 2 picks up the message and continues with its
processing.
We are seeing intermittent issues when processing large load of messages for example one
scenario is:
When I post a large number (20,000) of messages to queue 1 I see errors talking about a
failure while posting the message to queue 2. After this 20,000 message run is complete
(with some failures) if I post another 1000 messages non of them appear in the queue (I
was checking the JBM tables to verify this). At this point logs on both the nodes are
filled with debug and warn messages shown below, and messages wont get posted/processed.
If I kill node 2 (shutdown wont work) then all the 1000 messages get posted to Queue and
they all get processed on node1.
We had similar problems before and we changed the remoting component to version 2.2.3 SP2.
But we still have these problems. Any advice on what should we look in to will be greatly
helpful.
Thanks for your time.
Debug/Warn message on posting the second load of 1000 messages
2010-09-16 15:21:55,983 DEBUG org.jboss.remoting.transport.bisocket.BisocketServerInvoker
org.jboss.remoting.transport.bisocket.BisocketServerInvoker received new control socket
for unrecognized listenerId: c3l2mo-kgijx1-ge5qqvbw-1-ge5t0s2s-uq22010-09-16 15:21:56,000
DEBUG org.jboss.remoting.transport.bisocket.BisocketServerInvoker
org.jboss.remoting.transport.bisocket.BisocketServerInvoker received new control socket
for unrecognized listenerId: c3l2mo-kgijx1-ge5qqvbw-1-ge5s845p-7sx
2010-09-16 15:22:20,605 WARN org.jboss.remoting.transport.bisocket.BisocketServerInvoker
org.jboss.remoting.transport.bisocket.BisocketServerInvoker
org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@4f2ea142:
detected failure on control connection Thread[control:
Socket[addr=/12.129.94.24,port=43136,localport=55901],5,jboss]
(c3l2mp-ougfh2-ge5qshw2-1-ge5saop5-ct1: requesting new control connection
ERROR form the log while processing the initial large load (20,000)
org.jboss.jms.exception.MessagingJMSException: Failed to invoke at
org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:271) at
org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:205) at
org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160) at
org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(ClientSessionDelegate.java:499)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send_6145266547759487588.java)
at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661) at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect28.invoke(SessionAspect28.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send_6145266547759487588.java)
at
org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:114)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105) at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send_6145266547759487588.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170) at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105) at
org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeNext(ClientSessionDelegate$send_6145266547759487588.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java) at
org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:276) at
org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect39.invoke(ProducerAspect39.java)
at
org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(ClientProducerDelegate$send_3961598017717988886.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170) at
org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105) at
org.jboss.jms.client.delegate.ClientProducerDelegate$send_3961598017717988886.invokeNext(ClientProducerDelegate$send_3961598017717988886.java)
at org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)
at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:165) at
org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:208) at
org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:146) at
com.liaison.merckContivoPilot.MDBClients.DeltaThreadMDBClient.writeToQueue(DeltaThreadMDBClient.java:120)
at
com.liaison.merckContivoPilot.bo.SDTMTransformer.performTransformation(SDTMTransformer.java:317)
at
com.liaison.merckContivoPilot.ejb.TransformThreadMDB.onMessage(TransformThreadMDB.java:80)
at sun.reflect.GeneratedMethodAccessor128.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at
java.lang.reflect.Method.invoke(Method.java:597) at
org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112) at
org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
at
org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101) at
org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at
org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.aspects.tx.TxPolicy.invokeInCallerTx(TxPolicy.java:126)
at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:195)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at
org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.mdb.MessagingContainer.localInvoke(MessagingContainer.java:249)
at
org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.delivery(MessageInflowLocalProxy.java:268)
at
org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:138)
at $Proxy84.onMessage(Unknown Source)
at
org.jboss.resource.adapter.jms.inflow.JmsServerSession.onMessage(JmsServerSession.java:178)
at
org.jboss.jms.client.container.ClientConsumer.callOnMessageStatic(ClientConsumer.java:160)
at org.jboss.jms.client.container.SessionAspect.handleRun(SessionAspect.java:831)
at
org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect14.invoke(SessionAspect14.java)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_N8003352271541955702.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
at
org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_N8003352271541955702.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.run(ClientSessionDelegate.java)
at org.jboss.jms.client.JBossSession.run(JBossSession.java:199)
at org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:237)
at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:204)
at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:275)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:761)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalStateException: Cannot find object in dispatcher with id
8i8-3sy8s5eg-1-wbvqq5eg-1xjigk-om2l3c
at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:101)
at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:862)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:634)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:434)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:176)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:182)
at org.jboss.remoting.Client.invoke(Client.java:1935)
at org.jboss.remoting.Client.invoke(Client.java:788)
at org.jboss.remoting.Client.invoke(Client.java:776)
at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/562263#562263]
Start a new discussion in JBoss Messaging at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]