[jboss-user] [JBoss Messaging Users] - Failure on control connection Thread?
yongz
do-not-reply at jboss.com
Mon Nov 30 04:14:34 EST 2009
Hi,
* JBoss 5.0.1.GA
* JBoss messaging 1.4.6.GA
* JBoss remoting 2.5.2
* Java jdk1.5.0_19
We have a cluster with two nodes using clustered queue, and recently we are getting the following errors with JBoss messaging:
1. The following appears in Node 1:
| 2009-11-30 16:21:47,525 [Timer-6] WARN remoting.transport.bisocket.BisocketServerInvoker : org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask at 10d107: detected failure on control connection Thread[control: Socket[addr=myhost2/172.32.29.21,port=32062,localport=1881],5,] (4sl3l3l-7k73nm-g2mqo1ic-1-g2mqxt8r-jn: requesting new control connection
|
2. After a few minutes, on node 2[172.32.29.21]:
| 2009-11-30 16:24:45,243 [Timer-3] WARN remoting.transport.bisocket.BisocketServerInvoker : org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask at 17663eb: detected failure on control connection Thread[control: Socket[addr=myhost2/172.32.29.20,port=18861,localport=1686],5,] (4sl3l3k-n6bz0c-g2mqho1y-1-g2mqqj8h-j6: requesting new control connection
| 2009-11-30 16:24:49,886 [Timer-7] WARN remoting.transport.bisocket.BisocketClientInvoker : Unable to send ping: shutting down PingTimerTask
| java.net.SocketException: Software caused connection abort: socket write error
| at java.net.SocketOutputStream.socketWrite0(Native Method)
| at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
| at java.net.SocketOutputStream.write(SocketOutputStream.java:115)
| at org.jboss.remoting.transport.bisocket.BisocketClientInvoker$PingTimerTask.run(BisocketClientInvoker.java:723)
| at java.util.TimerThread.mainLoop(Timer.java:512)
| at java.util.TimerThread.run(Timer.java:462)
| 2009-11-30 16:25:45,864 [WorkerThread#0[172.21.129.129:1995]] ERROR jboss.messaging.util.ExceptionUtil : SessionEndpoint[l31-8c8rsm2g-1-y1ohqm2g-c0zb6n-k3l3ls4] createConsumerDelegate [m31-oc8rsm2g-1-y1ohqm2g-c0zb6n-k3l3ls4]
| javax.jms.InvalidDestinationException: No such destination: JBossTemporaryQueue[wj-xktxqm2g-1-ci1oqm2g-mn37k7-l3l3ls4] has it been deployed?
| at org.jboss.jms.server.endpoint.ServerSessionEndpoint.createConsumerDelegateInternal(ServerSessionEndpoint.java:1891)
| at org.jboss.jms.server.endpoint.ServerSessionEndpoint.createConsumerDelegate(ServerSessionEndpoint.java:262)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$createConsumerDelegate$aop(SessionAdvised.java:94)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised$createConsumerDelegate_8721389917985689973.invokeTarget(SessionAdvised$createConsumerDelegate_8721389917985689973.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
| at org.jboss.jms.server.container.SecurityAspect.handleCreateConsumerDelegate(SecurityAspect.java:123)
| at sun.reflect.GeneratedMethodAccessor3253.invoke(Unknown Source)
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:585)
| at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised.createConsumerDelegate(SessionAdvised.java)
| at org.jboss.jms.wireformat.SessionCreateConsumerDelegateRequest.serverInvoke(SessionCreateConsumerDelegateRequest.java:100)
| at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)
| at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:897)
| at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:768)
| at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:721)
| at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:575)
| at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
| 2009-11-30 16:30:55,300 [Timer-3] WARN remoting.transport.bisocket.BisocketServerInvoker : org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask at 17663eb: detected failure on control connection Thread[control: Socket[addr=myhost1/172.32.29.20,port=18861,localport=1843],5,] (4sl3l3k-n6bz0c-g2mqho1y-1-g2mqqj8h-j6: requesting new control connection
|
3. Then on node 1:
| 2009-11-30 16:25:45,706 [Thread-75] ERROR jboss.jms.client.FailoverCommandCenter : Failover failed
| javax.jms.InvalidDestinationException: No such destination: JBossTemporaryQueue[wj-xktxqm2g-1-ci1oqm2g-mn37k7-l3l3ls4] has it been deployed?
| at org.jboss.jms.server.endpoint.ServerSessionEndpoint.createConsumerDelegateInternal(ServerSessionEndpoint.java:1891)
| at org.jboss.jms.server.endpoint.ServerSessionEndpoint.createConsumerDelegate(ServerSessionEndpoint.java:262)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$createConsumerDelegate$aop(SessionAdvised.java:94)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised$createConsumerDelegate_8721389917985689973.invokeTarget(SessionAdvised$createConsumerDelegate_8721389917985689973.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
| at org.jboss.jms.server.container.SecurityAspect.handleCreateConsumerDelegate(SecurityAspect.java:123)
| at sun.reflect.GeneratedMethodAccessor3253.invoke(Unknown Source)
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:585)
| at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised.createConsumerDelegate(SessionAdvised.java)
| at org.jboss.jms.wireformat.SessionCreateConsumerDelegateRequest.serverInvoke(SessionCreateConsumerDelegateRequest.java:100)
| at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)
| at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:897)
| at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:768)
| at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:721)
| at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:575)
| at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
| at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:213)
| at org.jboss.remoting.Client.invoke(Client.java:1917)
| at org.jboss.remoting.Client.invoke(Client.java:768)
| at org.jboss.remoting.Client.invoke(Client.java:756)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$createConsumerDelegate$aop(ClientSessionDelegate.java:267)
| at org.jboss.jms.client.delegate.ClientSessionDelegate$createConsumerDelegate_8721389917985689973.invokeTarget(ClientSessionDelegate$createConsumerDelegate_8721389917985689973.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
| at org.jboss.jms.client.container.StateCreationAspect.handleCreateConsumerDelegate(StateCreationAspect.java:136)
| at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect_z_handleCreateConsumerDelegate_13797435.invoke(StateCreationAspect_z_handleCreateConsumerDelegate_13797435.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.ConsumerAspect.handleCreateConsumerDelegate(ConsumerAspect.java:76)
| at org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect_z_handleCreateConsumerDelegate_13797435.invoke(ConsumerAspect_z_handleCreateConsumerDelegate_13797435.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:114)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.createConsumerDelegate(ClientSessionDelegate.java)
| at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:249)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:128)
| at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:177)
| at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:129)
| at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:145)
| at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
| at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:88)
| at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:498)
|
| org.jboss.jms.exception.MessagingJMSException: Failed to invoke
| at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:271)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:205)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvokeOneway(DelegateSupport.java:165)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(ClientSessionDelegate.java:495)
| at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeTarget(ClientSessionDelegate$send_6145266547759487588.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
| at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)
| at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect_z_handleSend_13797435.invoke(SessionAspect_z_handleSend_13797435.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:114)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)
| at org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:276)
| at org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect_z_handleSend_13797435.invoke(ProducerAspect_z_handleSend_13797435.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)
| at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:165)
| at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:208)
| at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:146)
| ...............................
|
| at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:260)
| at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)
| at java.lang.Thread.run(Thread.java:595)
| Caused by: java.lang.Exception: Can not make remoting client invocation due to not being connected to server.
| at org.jboss.remoting.Client.invoke(Client.java:1922)
| at org.jboss.remoting.Client.invoke(Client.java:768)
| at org.jboss.remoting.Client.invokeOneway(Client.java:817)
| at org.jboss.remoting.Client.invokeOneway(Client.java:1003)
| at org.jboss.remoting.Client.invokeOneway(Client.java:994)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:180)
| ... 102 more
| 2009-11-30 16:25:45,815 [Thread-75] ERROR jms.client.container.ConnectionFailureListener : Caught exception in handling failure
| javax.jms.InvalidDestinationException: No such destination: JBossTemporaryQueue[wj-xktxqm2g-1-ci1oqm2g-mn37k7-l3l3ls4] has it been deployed?
| at org.jboss.jms.server.endpoint.ServerSessionEndpoint.createConsumerDelegateInternal(ServerSessionEndpoint.java:1891)
| at org.jboss.jms.server.endpoint.ServerSessionEndpoint.createConsumerDelegate(ServerSessionEndpoint.java:262)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$createConsumerDelegate$aop(SessionAdvised.java:94)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised$createConsumerDelegate_8721389917985689973.invokeTarget(SessionAdvised$createConsumerDelegate_8721389917985689973.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
| at org.jboss.jms.server.container.SecurityAspect.handleCreateConsumerDelegate(SecurityAspect.java:123)
| at sun.reflect.GeneratedMethodAccessor3253.invoke(Unknown Source)
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:585)
| at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.server.endpoint.advised.SessionAdvised.createConsumerDelegate(SessionAdvised.java)
| at org.jboss.jms.wireformat.SessionCreateConsumerDelegateRequest.serverInvoke(SessionCreateConsumerDelegateRequest.java:100)
| at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)
| at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:897)
| at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:768)
| at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:721)
| at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:575)
| at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
| at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:213)
| at org.jboss.remoting.Client.invoke(Client.java:1917)
| at org.jboss.remoting.Client.invoke(Client.java:768)
| at org.jboss.remoting.Client.invoke(Client.java:756)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
| at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$createConsumerDelegate$aop(ClientSessionDelegate.java:267)
| at org.jboss.jms.client.delegate.ClientSessionDelegate$createConsumerDelegate_8721389917985689973.invokeTarget(ClientSessionDelegate$createConsumerDelegate_8721389917985689973.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
| at org.jboss.jms.client.container.StateCreationAspect.handleCreateConsumerDelegate(StateCreationAspect.java:136)
| at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect_z_handleCreateConsumerDelegate_13797435.invoke(StateCreationAspect_z_handleCreateConsumerDelegate_13797435.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.ConsumerAspect.handleCreateConsumerDelegate(ConsumerAspect.java:76)
| at org.jboss.aop.advice.org.jboss.jms.client.container.ConsumerAspect_z_handleCreateConsumerDelegate_13797435.invoke(ConsumerAspect_z_handleCreateConsumerDelegate_13797435.java)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:114)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
| at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
| at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.createConsumerDelegate(ClientSessionDelegate.java)
| at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:249)
| at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:128)
| at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:177)
| at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:129)
| at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:145)
| at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
| at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:88)
| at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:498)
|
We saw the similar issue with JBoss messaing 1.4.2.SP1, but still got it with JBoss messaging 1.4.6, as a result, the two nodes lost connection with each other, can you please advise what could be the cause? Thanks in advance.
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4268030#4268030
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4268030
More information about the jboss-user
mailing list