Hi All,
I have an issue with the JBoss Cache which we are using as part of Apache Service Mix on a Cluster Environment.
We started the service mix process on Cluster environment and tried to trigger some operations such as topology upload which tries to execute an get request operations on the Network Element using the Mediation Framework on a Cluster Environment where it tries to acquire a lock on Cache for each operation triggered when it needs to write some content to Cache.
But we observed some issues with this operation trigger which is related to the JBoss Cache.
When we triggered the upload operations it tries to acquire a lock on the Cache to write some content but we are getting the exception’s as below:
2011-03-17T05:42:42.164+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22544 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000001502, reason :: Unable to acquire lock on Fqn [/jboss/AgentInfo] after [10000] milliseconds for requestor [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22544,5,main]]! Lock held by [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,main]]
2011-03-17T05:43:43.105+0200 | fifur | ERROR | NE3SWS_MEDIATION_Worker-1 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000003241, reason :: Unable to acquire lock on Fqn [/jboss/AgentInfo] after [10000] milliseconds for requestor [Thread[NE3SWS_MEDIATION_Worker-1,4,main]]! Lock held by [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,]]
2011-03-17T05:44:21.852+0200 | fifur | ERROR | NE3SWS_MEDIATION_Worker-1 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000003249, reason :: Unable to acquire lock on Fqn [/jboss/AgentInfo] after [10000] milliseconds for requestor [Thread[NE3SWS_MEDIATION_Worker-1,4,main]]! Lock held by [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,]]
Case1
So if we see the above logs all the new operations triggered are failing to acquire the lock because Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 is holding the lock, So we started checking the status of the thread Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 and we could never see a thread initiated as part of this process.
Case2
Sometimes we were able to find the thread which is holding the lock was terminated with SQL Exception as below
011-03-17T05:44:54.822+0200 | fifur | ERROR | pool-flow.seda.servicemix-scripting-thread-6271 | ScriptingEndpoint | ix.scripting.ScriptingEndpoint 309 | Error executing the script: null at line: -1 and column: -1
javax.script.ScriptException: org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/jboss/AU_STATUS_CACHETYPE] after [10000] milliseconds for requestor [Thread[pool-flow.seda.servicemix-scripting-thread-6271,5,main]]! Lock held by [Thread[pool-flow.seda.servicemix-scripting-thread-6267,5,]]
Here we could see that the lock was held by the thread Thread[pool-flow.seda.servicemix-scripting-thread-6267, and we can see from the below logs that the thread was terminated with SQL Exception
2011-03-17T05:41:44.820+0200 | fifur | INFO | pool-flow.seda.servicemix-scripting-thread-6267 | pinger | ect.GeneratedMethodAccessor149 ? | Getting all data from cache
2011-03-17T05:42:32.129+0200 | fifur | WARN | pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 | ManagedConnectionFactory | oader.ManagedConnectionFactory 146 | Failed to close connection
java.sql.SQLException: Already closed.
at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:114)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:191)
at org.jboss.cache.loader.ManagedConnectionFactory.safeClose(ManagedConnectionFactory.java:142)
at org.jboss.cache.loader.ManagedConnectionFactory.close(ManagedConnectionFactory.java:129)
at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoader.java:427)
at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDelegatingCacheLoader.java:94)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:314)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(CacheLoaderInterceptor.java:179)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)
at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:130)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)
at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocationDelegate.java:494)
at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocationDelegate.java:365)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProcessor.java:228)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieve(JbossCacheProcessor.java:103)
at com.nsn.oss.mf.mediation.ne3sws.ExchangeUtils.retrieveAgentInfo(ExchangeUtils.java:177)
at com.nsn.oss.mf.mediation.ne3sws.NE3SWSNotificationEndpoint.updateAgentInfo(NE3SWSNotificationEndpoint.java:460)
at com.nsn.oss.mf.mediation.ne3sws.NE3SWSNotificationEndpoint.processInOut(NE3SWSNotificationEndpoint.java:186)
at org.apache.servicemix.common.endpoints.ProviderEndpoint.process(ProviderEndpoint.java:111)
at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:632)
at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:185)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2011-03-17T05:42:32.131+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000001502, reason :: java.sql.SQLException: Io exception: Connection reset
2011-03-17T05:42:32.131+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 | NE3SWS_MEDIATION | sws.NE3SWSNotificationEndpoint 388 | Fault thrown to agent, message :: Agent not registered fault code :: Client.NotRegistered reason ::Client.NotRegistered
2011-03-17T05:42:36.075+0200 | fifur | WARN | pool-flow.seda.servicemix-scripting-thread-6267 | ManagedConnectionFactory | oader.ManagedConnectionFactory 146 | Failed to close connection
java.sql.SQLException: Already closed.
at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:114)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:191)
at org.jboss.cache.loader.ManagedConnectionFactory.safeClose(ManagedConnectionFactory.java:142)
at org.jboss.cache.loader.ManagedConnectionFactory.close(ManagedConnectionFactory.java:129)
at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoader.java:427)
at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDelegatingCacheLoader.java:94)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:314)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(CacheLoaderInterceptor.java:179)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)
at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:130)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)
at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocationDelegate.java:494)
at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocationDelegate.java:365)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProcessor.java:228)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCacheProcessor.java:122)
at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
at Script1.run(Script1.groovy:26)
at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:243)
at com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.java:46)
at javax.script.CompiledScript.eval(Unknown Source)
at org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpoint.java:300)
at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:632)
at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:185)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2011-03-17T05:42:36.077+0200 | fifur | ERROR | pool-flow.seda.servicemix-scripting-thread-6267 | ScriptingEndpoint | ix.scripting.ScriptingEndpoint 309 | Error executing the script: null at line: -1 and column: -1
javax.script.ScriptException: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset
at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:245)
at com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.java:46)
at javax.script.CompiledScript.eval(Unknown Source)
at org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpoint.java:300)
at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:632)
at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:185)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:304)
at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocationDelegate.java:494)
at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocationDelegate.java:365)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProcessor.java:228)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCacheProcessor.java:122)
at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
at Script1.run(Script1.groovy:26)
at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:243)
... 14 more
Caused by: java.sql.SQLException: Io exception: Connection reset
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:826)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1039)
at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:850)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1134)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3339)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3384)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoader.java:416)
at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDelegatingCacheLoader.java:94)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:314)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(CacheLoaderInterceptor.java:179)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)
at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:130)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)
... 30 more
2011-03-17T05:42:36.079+0200 | fifur | ERROR | pool-flow.seda.servicemix-scripting-thread-6267 | ScriptingComponent | emix.common.AsyncBaseLifeCycle 537 | Error processing exchange InOnly[
id: ID:10.8.74.54-12ea59a53d1-16:10312
status: Active
role: provider
service: {http://servicemix.apache.org/test}pinger_script
endpoint: endpoint
in: <?xml version="1.0" encoding="UTF-8"?><timer><name>{http://servicemix.apache.org/test}au.generate_ping.scheduler:endpoint</name><group>DEFAULT</group><fullname>DEFAULT.{http://servicemix.apache.org/test}au.generate_ping.scheduler:endpoint</fullname><description/><fireTime>Thu Mar 17 05:41:44 EET 2011</fireTime></timer>
]
javax.script.ScriptException: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset
at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:245)
at com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.java:46)
at javax.script.CompiledScript.eval(Unknown Source)
at org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpoint.java:300)
at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:632)
at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:185)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:304)
at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocationDelegate.java:494)
at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocationDelegate.java:365)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProcessor.java:228)
at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCacheProcessor.java:122)
at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
at Script1.run(Script1.groovy:26)
at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:243)
... 14 more
Caused by: java.sql.SQLException: Io exception: Connection reset
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:826)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1039)
at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:850)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1134)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3339)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3384)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoader.java:416)
at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDelegatingCacheLoader.java:94)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:314)
at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(CacheLoaderInterceptor.java:179)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)
at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:130)
at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVisitor.java:115)
at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeCommand.java:76)
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)
... 30 more
2011-03-17T05:42:42.164+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22544 |
Regards
Guruprasad BH