Environment
-------------------------
JBoss 4.2.3
JBoss Messaging 1.4.4
JBoss Remoting 2.2.3
We have a couple servers and each have local queues and remote mdbs. After running for a couple of hours, we see the following exception - which mentions 'marshalling exception'. The problem is - after this exception, the remote mdbs for that server never seem to get triggered. We have to restart the JBoss server in order to the MDBs triggered.
{noformat}
2010-06-11 11:05:18,777 ERROR [org.jboss.remoting.transport.socket.SocketClientInvoker] Got marshalling exception, exiting
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:197)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:229)
at java.io.BufferedInputStream.read(BufferedInputStream.java:246)
at java.io.FilterInputStream.read(FilterInputStream.java:89)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1077)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:682)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:474)
at org.jboss.remoting.Client.setupClientLease(Client.java:1912)
at org.jboss.remoting.Client.connect(Client.java:1800)
at org.jboss.remoting.Client.connect(Client.java:652)
at org.jboss.jms.client.remoting.JMSRemotingConnection$1.run(JMSRemotingConnection.java:362)
at java.security.AccessController.doPrivileged(AccessController.java:246)
at org.jboss.jms.client.remoting.JMSRemotingConnection.start(JMSRemotingConnection.java:356)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:165)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:80)
at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
at org.jboss.jms.client.JBossConnectionFactory.createConnectionInternal(JBossConnectionFactory.java:205)
at org.jboss.jms.client.JBossConnectionFactory.createQueueConnection(JBossConnectionFactory.java:101)
at org.jboss.jms.client.JBossConnectionFactory.createQueueConnection(JBossConnectionFactory.java:95)
at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:187)
at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:260)
at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:188)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerStartDelivery(JMSContainerInvoker.java:615)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl$ExceptionListenerRunnable.run(JMSContainerInvoker.java:1533)
at java.lang.Thread.run(Thread.java:810)
2010-06-11 11:05:18,778 DEBUG [org.jboss.remoting.InvokerRegistry] removed SocketClientInvoker[1d2c1d2c, bisocket://cholpos01p.na.odcorp.net:14457] from registry
2010-06-11 11:05:18,778 DEBUG [org.jboss.remoting.transport.socket.MicroSocketClientInvoker] SocketClientInvoker[1d2c1d2c, bisocket://cholpos01p.na.odcorp.net:14457] disconnecting ...
2010-06-11 11:05:18,778 DEBUG [org.jboss.remoting.Client] Client[302125570:a1jn4v-s8ubem-gaai3x6d-1-gab58h4q-2yw] is disconnected
2010-06-11 11:05:18,778 DEBUG [org.jboss.remoting.Client] Client[376575602:a1jn4v-s8ubem-gaai3x6d-1-gab58h4q-2yx] is disconnected
2010-06-11 11:05:18,778 DEBUG [org.jboss.ejb.plugins.jms.DLQHandler] Initialization failed DLQHandler
org.jboss.jms.exception.MessagingNetworkFailureException
at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:240)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:198)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:80)
at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
at org.jboss.jms.client.JBossConnectionFactory.createConnectionInternal(JBossConnectionFactory.java:205)
at org.jboss.jms.client.JBossConnectionFactory.createQueueConnection(JBossConnectionFactory.java:101)
at org.jboss.jms.client.JBossConnectionFactory.createQueueConnection(JBossConnectionFactory.java:95)
at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:187)
at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:260)
at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:188)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerStartDelivery(JMSContainerInvoker.java:615)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl$ExceptionListenerRunnable.run(JMSContainerInvoker.java:1533)
at java.lang.Thread.run(Thread.java:810)
Caused by:
org.jboss.remoting.CannotConnectException: Error setting up client lease upon performing connect.
at org.jboss.remoting.Client.connect(Client.java:1804)
at org.jboss.remoting.Client.connect(Client.java:652)
at org.jboss.jms.client.remoting.JMSRemotingConnection$1.run(JMSRemotingConnection.java:362)
at java.security.AccessController.doPrivileged(AccessController.java:246)
at org.jboss.jms.client.remoting.JMSRemotingConnection.start(JMSRemotingConnection.java:356)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:165)
... 14 more
Caused by:
java.lang.Exception: Error setting up client lease
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:508)
at org.jboss.remoting.Client.setupClientLease(Client.java:1912)
at org.jboss.remoting.Client.connect(Client.java:1800)
... 19 more
Caused by:
java.rmi.MarshalException: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
java.net.SocketException: Connection reset
at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:127)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:740)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:474)
... 21 more
Caused by:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:197)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:229)
at java.io.BufferedInputStream.read(BufferedInputStream.java:246)
at java.io.FilterInputStream.read(FilterInputStream.java:89)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1077)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:682)
... 24 more
2010-06-11 11:05:18,778 ERROR [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Reconnect failed: JMS provider failure detected for ImportMerchandiseMDB
org.jboss.jms.exception.MessagingNetworkFailureException
at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:240)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:198)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.container.StateCreationAspect.handleCreateConnectionDelegate(StateCreationAspect.java:80)
at org.jboss.aop.advice.org.jboss.jms.client.container.StateCreationAspect0.invoke(StateCreationAspect0.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.invokeNext(ClientConnectionFactoryDelegate$createConnectionDelegate_N3019492359065420858.java)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.createConnectionDelegate(ClientConnectionFactoryDelegate.java)
at org.jboss.jms.client.JBossConnectionFactory.createConnectionInternal(JBossConnectionFactory.java:205)
at org.jboss.jms.client.JBossConnectionFactory.createQueueConnection(JBossConnectionFactory.java:101)
at org.jboss.jms.client.JBossConnectionFactory.createQueueConnection(JBossConnectionFactory.java:95)
at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:187)
at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:260)
at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:188)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerStartDelivery(JMSContainerInvoker.java:615)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker$ExceptionListenerImpl$ExceptionListenerRunnable.run(JMSContainerInvoker.java:1533)
at java.lang.Thread.run(Thread.java:810)
Caused by:
org.jboss.remoting.CannotConnectException: Error setting up client lease upon performing connect.
at org.jboss.remoting.Client.connect(Client.java:1804)
at org.jboss.remoting.Client.connect(Client.java:652)
at org.jboss.jms.client.remoting.JMSRemotingConnection$1.run(JMSRemotingConnection.java:362)
at java.security.AccessController.doPrivileged(AccessController.java:246)
at org.jboss.jms.client.remoting.JMSRemotingConnection.start(JMSRemotingConnection.java:356)
at org.jboss.jms.client.delegate.ClientConnectionFactoryDelegate.org$jboss$jms$client$delegate$ClientConnectionFactoryDelegate$createConnectionDelegate$aop(ClientConnectionFactoryDelegate.java:165)
... 14 more
Caused by:
java.lang.Exception: Error setting up client lease
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:508)
at org.jboss.remoting.Client.setupClientLease(Client.java:1912)
at org.jboss.remoting.Client.connect(Client.java:1800)
... 19 more
Caused by:
java.rmi.MarshalException: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
java.net.SocketException: Connection reset
at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:127)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:740)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:458)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:141)
at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:474)
... 21 more
Caused by:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:197)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:229)
at java.io.BufferedInputStream.read(BufferedInputStream.java:246)
at java.io.FilterInputStream.read(FilterInputStream.java:89)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:1077)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:682)
... 24 more
2010-06-11 11:05:18,778 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] innerStop
2010-06-11 11:05:18,778 DEBUG [org.jboss.jms.asf.StdServerSessionPool] Clearing 0 from ServerSessionPool
2010-06-11 11:05:18,778 DEBUG [org.jboss.ejb.plugins.jms.DLQHandler] Destroying DLQHandler
2010-06-11 11:05:18,778 DEBUG [org.jboss.ejb.plugins.jms.DLQHandler] Destroyed DLQHandler
2010-06-11 11:05:18,778 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Waiting for reconnect internal 10000ms for ImportMerchandiseMDB
2010-06-11 11:05:28,778 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Trying to reconnect to JMS provider for ImportMerchandiseMDB
2010-06-11 11:05:28,778 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Initializing
2010-06-11 11:05:28,779 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Looking up provider adapter: java:/RemoteJMSProvider
2010-06-11 11:05:28,779 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Provider adapter: org.jboss.jms.jndi.JNDIProviderAdapter@13ae13ae
2010-06-11 11:05:28,779 DEBUG [org.jboss.ejb.plugins.jms.DLQHandler] Creating DLQHandler
2010-06-11 11:05:28,956 DEBUG [org.jboss.ejb.plugins.jms.DLQHandler] Using factory: JBossConnectionFactory->ConnectionFactoryDelegate[jboss.messaging.connectionfactory:service=ConnectionFactory, SID=0]
{noformat}
Looking for suggestions on this. Customer is considering migrating back to MQ from Messaging because we didn't see these types of connection issues.