[jbosscache-issues] Query on JBoss Cache Lock issue on Cluster Environment !!

Friday, 25 March 2011

Hi All,

I have an issue with the JBoss Cache which we are using as part of
Apache Service Mix on a Cluster Environment.

We started the service mix process on Cluster environment and tried to
trigger some operations such as topology upload which tries to execute
an get request operations on the Network Element using the Mediation
Framework on a Cluster Environment where it tries to acquire a lock on
Cache for each operation triggered when it needs to write some content
to Cache.

But we observed some issues with this operation trigger which is related
to the JBoss Cache. 
When we triggered the upload operations it tries to acquire a lock on
the Cache to write some content but we are getting the exception's as
below:

2011-03-17T05:42:42.164+0200 | fifur | ERROR |
pool-flow.seda.ne3sws-mediation-notification-se-thread-22544 |
ExchangeUtils            | mediation.ne3sws.ExchangeUtils  179 | Error
while retrieving the agentInfo null agent unique id ::
6000160000000001502, reason :: Unable to acquire lock on Fqn
[/jboss/AgentInfo] after [10000] milliseconds for requestor
[Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22544,5,m
ain]]! Lock held by
[Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,m
ain]]

2011-03-17T05:43:43.105+0200 | fifur | ERROR | NE3SWS_MEDIATION_Worker-1
| ExchangeUtils            | mediation.ne3sws.ExchangeUtils  179 | Error
while retrieving the agentInfo null agent unique id ::
6000160000000003241, reason :: Unable to acquire lock on Fqn
[/jboss/AgentInfo] after [10000] milliseconds for requestor
[Thread[NE3SWS_MEDIATION_Worker-1,4,main]]! Lock held by
[Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,]
]

2011-03-17T05:44:21.852+0200 | fifur | ERROR | NE3SWS_MEDIATION_Worker-1
| ExchangeUtils            | mediation.ne3sws.ExchangeUtils  179 | Error
while retrieving the agentInfo null agent unique id ::
6000160000000003249, reason :: Unable to acquire lock on Fqn
[/jboss/AgentInfo] after [10000] milliseconds for requestor
[Thread[NE3SWS_MEDIATION_Worker-1,4,main]]! Lock held by
[Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,]
]

Case1
So if we see the above logs all the new operations triggered are failing
to acquire the lock because
Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 is
holding the lock, So we started checking the status of the thread
Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 and
we could never see a thread initiated as part of this process.

Case2
Sometimes we were able to find the thread which is holding the lock was
terminated with SQL Exception as below

011-03-17T05:44:54.822+0200 | fifur | ERROR |
pool-flow.seda.servicemix-scripting-thread-6271 | ScriptingEndpoint
| ix.scripting.ScriptingEndpoint  309 | Error executing the script: null
at line: -1 and column: -1
javax.script.ScriptException: org.jboss.cache.lock.TimeoutException:
Unable to acquire lock on Fqn [/jboss/AU_STATUS_CACHETYPE] after [10000]
milliseconds for requestor
[Thread[pool-flow.seda.servicemix-scripting-thread-6271,5,main]]! Lock
held by [Thread[pool-flow.seda.servicemix-scripting-thread-6267,5,]]

Here we could see that the lock was held by the thread
Thread[pool-flow.seda.servicemix-scripting-thread-6267, and we can see
from the below logs that the thread was terminated with SQL Exception

2011-03-17T05:41:44.820+0200 | fifur | INFO  |
pool-flow.seda.servicemix-scripting-thread-6267 | pinger
| ect.GeneratedMethodAccessor149    ? | Getting all data from cache
2011-03-17T05:42:32.129+0200 | fifur | WARN  |
pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 |
ManagedConnectionFactory | oader.ManagedConnectionFactory  146 | Failed
to close connection
java.sql.SQLException: Already closed.
	at
org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java
:114)
	at
org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.clo
se(PoolingDataSource.java:191)
	at
org.jboss.cache.loader.ManagedConnectionFactory.safeClose(ManagedConnect
ionFactory.java:142)
	at
org.jboss.cache.loader.ManagedConnectionFactory.close(ManagedConnectionF
actory.java:129)
	at
org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa
der.java:427)
	at
org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele
gatingCacheLoader.java:94)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo
aderInterceptor.java:314)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(
CacheLoaderInterceptor.java:179)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt
erceptor.java:301)
	at
org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j
ava:283)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo
cationContextInterceptor.java:178)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(
InvocationContextInterceptor.java:130)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja
va:287)
	at
org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati
onDelegate.java:494)
	at
org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio
nDelegate.java:365)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc
essor.java:228)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieve(JbossCachePro
cessor.java:103)
	at
com.nsn.oss.mf.mediation.ne3sws.ExchangeUtils.retrieveAgentInfo(Exchange
Utils.java:177)
	at
com.nsn.oss.mf.mediation.ne3sws.NE3SWSNotificationEndpoint.updateAgentIn
fo(NE3SWSNotificationEndpoint.java:460)
	at
com.nsn.oss.mf.mediation.ne3sws.NE3SWSNotificationEndpoint.processInOut(
NE3SWSNotificationEndpoint.java:186)
	at
org.apache.servicemix.common.endpoints.ProviderEndpoint.process(Provider
Endpoint.java:111)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC
ycle.java:627)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas
eLifeCycle.java:581)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB
aseLifeCycle.java:535)
	at
org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync
LifeCycleWrapper.java:60)
	at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D
eliveryChannelImpl.java:632)
	at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j
ava:185)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java
:168)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1
34)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
	at java.lang.Thread.run(Unknown Source)
2011-03-17T05:42:32.131+0200 | fifur | ERROR |
pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 |
ExchangeUtils            | mediation.ne3sws.ExchangeUtils  179 | Error
while retrieving the agentInfo null agent unique id ::
6000160000000001502, reason :: java.sql.SQLException: Io exception:
Connection reset
2011-03-17T05:42:32.131+0200 | fifur | ERROR |
pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 |
NE3SWS_MEDIATION         | sws.NE3SWSNotificationEndpoint  388 | Fault
thrown to agent,  message :: Agent not registered fault code ::
Client.NotRegistered reason ::Client.NotRegistered
2011-03-17T05:42:36.075+0200 | fifur | WARN  |
pool-flow.seda.servicemix-scripting-thread-6267 |
ManagedConnectionFactory | oader.ManagedConnectionFactory  146 | Failed
to close connection
java.sql.SQLException: Already closed.
	at
org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java
:114)
	at
org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.clo
se(PoolingDataSource.java:191)
	at
org.jboss.cache.loader.ManagedConnectionFactory.safeClose(ManagedConnect
ionFactory.java:142)
	at
org.jboss.cache.loader.ManagedConnectionFactory.close(ManagedConnectionF
actory.java:129)
	at
org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa
der.java:427)
	at
org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele
gatingCacheLoader.java:94)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo
aderInterceptor.java:314)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(
CacheLoaderInterceptor.java:179)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt
erceptor.java:301)
	at
org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j
ava:283)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo
cationContextInterceptor.java:178)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(
InvocationContextInterceptor.java:130)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja
va:287)
	at
org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati
onDelegate.java:494)
	at
org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio
nDelegate.java:365)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc
essor.java:228)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCache
Processor.java:122)
	at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
	at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
	at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
	at
org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper
.java:766)
	at
org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.jav
a:754)
	at
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBy
tecodeAdapter.java:170)
	at Script1.run(Script1.groovy:26)
	at
com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24
3)
	at
com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.jav
a:46)
	at javax.script.CompiledScript.eval(Unknown Source)
	at
org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpo
int.java:300)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC
ycle.java:627)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas
eLifeCycle.java:581)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB
aseLifeCycle.java:535)
	at
org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync
LifeCycleWrapper.java:60)
	at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D
eliveryChannelImpl.java:632)
	at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j
ava:185)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java
:168)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1
34)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
	at java.lang.Thread.run(Unknown Source)
2011-03-17T05:42:36.077+0200 | fifur | ERROR |
pool-flow.seda.servicemix-scripting-thread-6267 | ScriptingEndpoint
| ix.scripting.ScriptingEndpoint  309 | Error executing the script: null
at line: -1 and column: -1
javax.script.ScriptException: org.jboss.cache.CacheException:
java.sql.SQLException: Io exception: Connection reset
	at
com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24
5)
	at
com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.jav
a:46)
	at javax.script.CompiledScript.eval(Unknown Source)
	at
org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpo
int.java:300)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC
ycle.java:627)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas
eLifeCycle.java:581)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB
aseLifeCycle.java:535)
	at
org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync
LifeCycleWrapper.java:60)
	at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D
eliveryChannelImpl.java:632)
	at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j
ava:185)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java
:168)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1
34)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: org.jboss.cache.CacheException: java.sql.SQLException: Io
exception: Connection reset
	at
org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja
va:304)
	at
org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati
onDelegate.java:494)
	at
org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio
nDelegate.java:365)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc
essor.java:228)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCache
Processor.java:122)
	at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
	at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
	at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
	at
org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper
.java:766)
	at
org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.jav
a:754)
	at
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBy
tecodeAdapter.java:170)
	at Script1.run(Script1.groovy:26)
	at
com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24
3)
	... 14 more
Caused by: java.sql.SQLException: Io exception: Connection reset
	at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:11
2)
	at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:14
6)
	at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:25
5)
	at
oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedSt
atement.java:826)
	at
oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.
java:1039)
	at
oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPrepared
Statement.java:850)
	at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.
java:1134)
	at
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePrepare
dStatement.java:3339)
	at
oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedSt
atement.java:3384)
	at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat
ingPreparedStatement.java:96)
	at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat
ingPreparedStatement.java:96)
	at
org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa
der.java:416)
	at
org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele
gatingCacheLoader.java:94)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo
aderInterceptor.java:314)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(
CacheLoaderInterceptor.java:179)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt
erceptor.java:301)
	at
org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j
ava:283)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo
cationContextInterceptor.java:178)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(
InvocationContextInterceptor.java:130)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja
va:287)
	... 30 more
2011-03-17T05:42:36.079+0200 | fifur | ERROR |
pool-flow.seda.servicemix-scripting-thread-6267 | ScriptingComponent
| emix.common.AsyncBaseLifeCycle  537 | Error processing exchange
InOnly[
  id: ID:10.8.74.54-12ea59a53d1-16:10312
  status: Active
  role: provider
  service: {http://servicemix.apache.org/test}pinger_script
  endpoint: endpoint
  in: <?xml version="1.0"
encoding="UTF-8"?><timer><name>{http://servicemi...
erate_ping.scheduler:endpoint</name><group>DEFAULT</group><fullname>DEFA
ULT.{http://servicemix.apache.org/test}au.generate_ping.scheduler:endpoi
nt</fullname><description/><fireTime>Thu Mar 17 05:41:44 EET
2011</fireTime></timer>
]
javax.script.ScriptException: org.jboss.cache.CacheException:
java.sql.SQLException: Io exception: Connection reset
	at
com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24
5)
	at
com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.jav
a:46)
	at javax.script.CompiledScript.eval(Unknown Source)
	at
org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpo
int.java:300)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC
ycle.java:627)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas
eLifeCycle.java:581)
	at
org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB
aseLifeCycle.java:535)
	at
org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync
LifeCycleWrapper.java:60)
	at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D
eliveryChannelImpl.java:632)
	at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j
ava:185)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java
:168)
	at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1
34)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: org.jboss.cache.CacheException: java.sql.SQLException: Io
exception: Connection reset
	at
org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja
va:304)
	at
org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati
onDelegate.java:494)
	at
org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio
nDelegate.java:365)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc
essor.java:228)
	at
com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCache
Processor.java:122)
	at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
	at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
	at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
	at
org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper
.java:766)
	at
org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.jav
a:754)
	at
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBy
tecodeAdapter.java:170)
	at Script1.run(Script1.groovy:26)
	at
com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24
3)
	... 14 more
Caused by: java.sql.SQLException: Io exception: Connection reset
	at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:11
2)
	at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:14
6)
	at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:25
5)
	at
oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedSt
atement.java:826)
	at
oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.
java:1039)
	at
oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPrepared
Statement.java:850)
	at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.
java:1134)
	at
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePrepare
dStatement.java:3339)
	at
oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedSt
atement.java:3384)
	at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat
ingPreparedStatement.java:96)
	at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat
ingPreparedStatement.java:96)
	at
org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa
der.java:416)
	at
org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele
gatingCacheLoader.java:94)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo
aderInterceptor.java:314)
	at
org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand(
CacheLoaderInterceptor.java:179)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt
erceptor.java:301)
	at
org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j
ava:283)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma
ndInterceptor.java:131)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept
or(CommandInterceptor.java:116)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo
cationContextInterceptor.java:178)
	at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault(
InvocationContextInterceptor.java:130)
	at
org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis
itor.java:115)
	at
org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman
d.java:76)
	at
org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja
va:287)
	... 30 more
2011-03-17T05:42:42.164+0200 | fifur | ERROR |
pool-flow.seda.ne3sws-mediation-notification-se-thread-22544 |

Regards
Guruprasad BH

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[jbosscache-issues] Query on JBoss Cache Lock issue on Cluster Environment !!