March 2011 - jbosscache-issues - Jboss List Archives

[JBoss JIRA] Created: (JBCACHE-1523) JBoss Cache MBean print* operations return Object's toString()

by Ondrej Žižka (JIRA)

JBoss Cache MBean print* operations return Object's toString() -------------------------------------------------------------- Key: JBCACHE-1523 URL: https://jira.jboss.org/jira/browse/JBCACHE-1523 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Reporter: Ondrej Žižka Assignee: Manik Surtani Priority: Minor 1) Run EAP 5, all config 2) Go to JMX console, jboss.cache:config=ha-partition,service=Cache 3) Invoke print* operations. Cache details: "/ null", as HTML /  null<br/> "Print configuration" returns only java.lang.Object's toString(), e.g. org.jboss.cache.config.Configuration@af91c6fa etc. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 10 months

3
4
0 / 0

[JBoss JIRA] Created: (JBCACHE-1410) Cache loaders that use StripedLock do not adequately lock root node when the root node is first added.

by Manik Surtani (JIRA)

Cache loaders that use StripedLock do not adequately lock root node when the root node is first added. ------------------------------------------------------------------------------------------------------ Key: JBCACHE-1410 URL: https://jira.jboss.org/jira/browse/JBCACHE-1410 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Components: Cache loaders Affects Versions: 2.0.0.GA Reporter: Manik Surtani Assignee: Manik Surtani Fix For: 3.0.0.GA, 2.X See detail on the forum post. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years

4
12
0 / 0

[JBoss JIRA] Created: (JBCACHE-1585) JBossCache eviction thread creating a Timer without making the new thread a daemon thread

by Mats Henrikson (JIRA)

JBossCache eviction thread creating a Timer without making the new thread a daemon thread ----------------------------------------------------------------------------------------- Key: JBCACHE-1585 URL: https://jira.jboss.org/browse/JBCACHE-1585 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Affects Versions: 1.4.1.SP13 Reporter: Mats Henrikson Assignee: Manik Surtani There is a problem where the class org.jboss.cache.eviction.RegionManager.EvictionTreeCacheListener.cacheStarted(), line 499: {code} evictionThread_ = new Timer(); {code} Since the new Timer() is created without telling it that it is a daemon thread this Timer thread will actually cause the JVM to not exit when the main() method of the application finishes - it makes it necessary to call System.exit() to shut the JVM down. Since this is the thread that periodically runs the specified eviction policy there is no need for it to cause the JVM to stay running - it's up to the developer to say if the JVM should stay running or not, not JBossCache. The line should be changed to something like: {code} evictionThread_ = new Timer(true); {code} in order to specify that this is a daemon thread. (At the same time it would also be good if the Timer is passed a thread name like for instance "JBossCache Eviction Thread".) Please see the JDK docs for more info: http://java.sun.com/j2se/1.5.0/docs/api/java/util/Timer.html#Timer(java.l... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years, 2 months

2
2
0 / 0

[JBoss JIRA] Created: (JBCACHE-1596) Initial state transfer with more than 200, 000 nodes deadlock on eviction queue

by Dennis Reed (JIRA)

Initial state transfer with more than 200,000 nodes deadlock on eviction queue ------------------------------------------------------------------------------ Key: JBCACHE-1596 URL: https://issues.jboss.org/browse/JBCACHE-1596 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Components: Eviction Affects Versions: 1.4.1.SP13 Reporter: Dennis Reed Assignee: Manik Surtani When the cache has more than 200,000 entries, state transfer deadlocks on the joining node, due to filling up and blocking the eviction queue before the eviction thread has been started. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years, 4 months

2
5
0 / 0

[JBoss JIRA] Created: (JBCACHE-1434) Fix UnitTestCacheFactory extractTestName method

by Dominik Pospisil (JIRA)

Fix UnitTestCacheFactory extractTestName method ----------------------------------------------- Key: JBCACHE-1434 URL: https://jira.jboss.org/jira/browse/JBCACHE-1434 Project: JBoss Cache Issue Type: Sub-task Security Level: Public (Everyone can see) Reporter: Dominik Pospisil Assignee: Dominik Pospisil UnitTestCacheFactory uses stack trace introspection to retreive executed test class name. It does not work if the factory method is called from a seperclass of test class. Find other way around to retrieve test class. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years, 5 months

1
3
0 / 0

[JBoss JIRA] Created: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster

by l jw (JIRA)

org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster -------------------------------------------------------------------------------------------------- Key: JBCACHE-1592 URL: https://jira.jboss.org/browse/JBCACHE-1592 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Components: Locking Affects Versions: 3.2.5.GA Environment: windows os,cluster node in one machine. Reporter: l jw Assignee: Manik Surtani Priority: Blocker When used in cluster,the following exception is throw frequently: org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/courseClass/360/male] after [5000] milliseconds for requestor [GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by [GlobalTransaction:<172.18.73.162:3428>:6416] at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159) at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217) at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186) at org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101) at org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88) at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150) at org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107) at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501) at org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388) at org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134) at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140) at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178) at org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106) at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68) at org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316) at org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139) at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115) at org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319) at org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246) at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637) at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545) at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368) at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775) at org.jgroups.JChannel.up(JChannel.java:1336) at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454) at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486) at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319) at org.jgroups.protocols.FRAG2.up(FRAG2.java:188) at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820) at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233) at org.jgroups.protocols.UNICAST.up(UNICAST.java:328) at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895) at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708) at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167) at org.jgroups.protocols.FD.up(FD.java:284) at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307) at org.jgroups.protocols.MERGE2.up(MERGE2.java:144) at org.jgroups.protocols.Discovery.up(Discovery.java:264) at org.jgroups.protocols.PING.up(PING.java:273) at org.jgroups.protocols.TP.passMessageUp(TP.java:1273) at org.jgroups.protocols.TP.access$100(TP.java:49) at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826) at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) and following is my config: <jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.2"> <locking isolationLevel="REPEATABLE_READ" lockParentForChildInsertRemove="false" lockAcquisitionTimeout="5000" nodeLockingScheme="mvcc" writeSkewCheck="true" useLockStriping="true" concurrencyLevel="10000"/> <clustering mode="replication" clusterName="JBossCache-cluster"> <stateRetrieval timeout="5000" fetchInMemoryState="false" nonBlocking="false"/> <sync replTimeout="5000"/> <jgroupsConfig> <UDP discard_incompatible_packets="true" enable_bundling="false" enable_diagnostics="false" ip_ttl="2" loopback="false" max_bundle_size="64000" max_bundle_timeout="30" mcast_addr="224.0.0.1" mcast_port="45588" mcast_recv_buf_size="25000000" mcast_send_buf_size="640000" oob_thread_pool.enabled="true" oob_thread_pool.keep_alive_time="10000" oob_thread_pool.max_threads="4" oob_thread_pool.min_threads="1" oob_thread_pool.queue_enabled="true" oob_thread_pool.queue_max_size="10" oob_thread_pool.rejection_policy="Run" thread_naming_pattern="pl" thread_pool.enabled="true" thread_pool.keep_alive_time="30000" thread_pool.max_threads="25" thread_pool.min_threads="1" thread_pool.queue_enabled="true" thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run" tos="8" ucast_recv_buf_size="20000000" ucast_send_buf_size="640000" use_concurrent_stack="true" use_incoming_packet_handler="true"/> <PING num_initial_members="3" timeout="2000"/> <MERGE2 max_interval="30000" min_interval="10000"/> <FD_SOCK/> <FD max_tries="5" shun="true" timeout="10000"/> <VERIFY_SUSPECT timeout="1500"/> <pbcast.NAKACK discard_delivered_msgs="true" gc_lag="0" retransmit_timeout="300,600,1200,2400,4800" use_mcast_xmit="false"/> <UNICAST timeout="300,600,1200,2400,3600"/> <pbcast.STABLE desired_avg_gossip="50000" max_bytes="400000" stability_delay="1000"/> <pbcast.GMS join_timeout="5000" print_local_addr="true" shun="false" view_ack_collection_timeout="5000" view_bundling="true"/> <FRAG2 frag_size="60000"/> <pbcast.STREAMING_STATE_TRANSFER/> <pbcast.FLUSH timeout="0"/> </jgroupsConfig> </clustering> </jbosscache> -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years, 5 months

3
5
0 / 0

[JBoss JIRA] Created: (JBCACHE-1582) ExpirationAlgorithm doesn't evict nodes that have been created in a long Tx

by Nicolas Filotto (JIRA)

ExpirationAlgorithm doesn't evict nodes that have been created in a long Tx ---------------------------------------------------------------------------- Key: JBCACHE-1582 URL: https://jira.jboss.org/browse/JBCACHE-1582 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Components: Eviction Affects Versions: 3.2.5.GA Reporter: Nicolas Filotto Assignee: Manik Surtani Attachments: TestExpirationAlgorithm.java, TestExpirationAlgorithm.java I noticed that I had a memory leak in my application when I use the ExpirationAlgorithm (maybe this bug happens with other algorithms) some nodes were never evicted. After a deeper investigation, I noticed that the no EvictionEntry was created for those nodes since the method getExpiration returns null (which also means that cache.peek(fqn, false) returns null). I finally found the problem, that is because if the Expiration Thread wakes up before that the related Tx is committed, it won't find the node into the cache so it will considered it as removed but in fact it is just because the Tx has not yet been committed. This has for consequence that the node will never be evicted and thus create a memory leak. Please find the related unit test as attached file. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years, 8 months

3
15
0 / 0

Query related to LockNotReleaseException (JGroups Issue) !!

by Bh, Guruprasad (NSN - IN/Bangalore)

Hi All, I have an issue with the Jgroups locking problem with the Service mix, here in the below logs you can see the Service Mix File Poller is trying to lock for the same key(same file) two times and fails. Because of this the unlock call is failing every time this happens, this scenario is sporadic but happens frequently. Scenario: First time the File Poller thread will pole the file and try's to acquire the lock on the same and it was granted successfully Second time it tries to lock same file and says lock was not granted, and the lock aquire process was failed But after the second failure of the locking on the same file it tries to unlock immediately as the lock was not granted, but even the unlock will fails saying unlocking is failed Then again the same file poller thread tries to unlock the lock for the same key and again it fails with exception as below: 2011-03-24T11:20:05.379+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 128 | Lock cannot be unlocked. org.jgroups.blocks.LockNotReleasedException: Lock cannot be unlocked. Here in all the above operations you can see that the thread id is same !! Do anyone has idea on this problem / issue ? 2011-03-24T11:20:05.182+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 92 | Getting lock object for key:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.182+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | DistributedLock | mf.lockmanager.DistributedLock 106 | Trying to obtain a lock with key:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.183+0200 | thridoo | INFO | pool-flow.seda.ne3s-ws-smi-bc-thread-358 | NE3SWS-BC | .ne3sws.endpoints.NE3SWSClient 294 | User credentials used for http://agent84:8971/NE3S/1.0/NE3SBasicOperationsService user :: wsuser password :: ***** 2011-03-24T11:20:05.202+0200 | thridoo | DEBUG | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-270 | NE3SWS_MEDIATION | mediation.ne3sws.NE3SWSManager 442 | oper time - null :: 34 2011-03-24T11:20:05.202+0200 | thridoo | INFO | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-270 | NE3SWS_MEDIATION | ediation.ne3sws.NE3SWSEndpoint 168 | Triggered the operation set successfully for agent agent84 user ::wsuser password :: ***** ne3sws endpoint ::http://agent84:8971/NE3S/1.0/NE3SBasicOperationsService operation id :: null time taken (in ms) :: 46. waiting for response from Agent 2011-03-24T11:20:05.203+0200 | thridoo | DEBUG | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-270 | NE3SWS_MEDIATION | ediation.ne3sws.NE3SWSEndpoint 239 | Excptn during exchg cleanup 2011-03-24T11:20:05.204+0200 | thridoo | DEBUG | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-271 | ClusteredLockManager | ckmanager.ClusteredLockManager 92 | Getting lock object for key:an_globalId_502116000000000569_2570905160234671.xml 2011-03-24T11:20:05.205+0200 | thridoo | DEBUG | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-271 | ClusteredLockManager | ckmanager.ClusteredLockManager 107 | Lock already available for the key:an_globalId_502116000000000569_2570905160234671.xml 2011-03-24T11:20:05.205+0200 | thridoo | DEBUG | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-271 | DistributedLock | mf.lockmanager.DistributedLock 149 | Releasing lock:an_globalId_502116000000000569_2570905160234671.xml 2011-03-24T11:20:05.250+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | DistributedLock | mf.lockmanager.DistributedLock 126 | Tried lock was granted:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.250+0200 | thridoo | DEBUG | pool-flow.seda.ne3sws-mediation-enhancer-se-thread-271 | ClusteredLockManager | ckmanager.ClusteredLockManager 121 | removing Distribute lock object for key:an_globalId_502116000000000569_2570905160234671.xml 2011-03-24T11:20:05.252+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 92 | Getting lock object for key:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.252+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 107 | Lock already available for the key:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.252+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | DistributedLock | mf.lockmanager.DistributedLock 149 | Releasing lock:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.281+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-74 | ClusteredLockManager | ckmanager.ClusteredLockManager 92 | Getting lock object for key:an_globalId_502116000000000594_2570887937874621.xml 2011-03-24T11:20:05.281+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-74 | DistributedLock | mf.lockmanager.DistributedLock 106 | Trying to obtain a lock with key:an_globalId_502116000000000594_2570887937874621.xml 2011-03-24T11:20:05.314+0200 | thridoo | WARN | pool-component.servicemix-file-thread-112 | DistributedLock | mf.lockmanager.DistributedLock 156 | Unlocking failed for lock:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.315+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 121 | removing Distribute lock object for key:an_globalId_502116000000000526_2567740520833480.xml 2011-03-24T11:20:05.347+0200 | thridoo | INFO | pool-component.servicemix-file-thread-74 | DistributedLock | mf.lockmanager.DistributedLock 116 | Lock (an_globalId_502116000000000594_2570887937874621.xml) could not be granted! 2011-03-24T11:20:05.378+0200 | thridoo | ERROR | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 126 | LockNotReleasedException during distributed lock release 2011-03-24T11:20:05.379+0200 | thridoo | DEBUG | pool-component.servicemix-file-thread-112 | ClusteredLockManager | ckmanager.ClusteredLockManager 128 | Lock cannot be unlocked. org.jgroups.blocks.LockNotReleasedException: Lock cannot be unlocked. at org.jgroups.blocks.DistributedLockManager.unlock(DistributedLockManager. java:311) at org.jgroups.blocks.DistributedLockManager.unlock(DistributedLockManager. java:253) at com.nsn.obs.mf.lockmanager.ClusteredLockManager.removeLock(ClusteredLock Manager.java:123) at org.apache.servicemix.file.FilePollerEndpoint.unlockAsyncFile(FilePoller Endpoint.java:481) at org.apache.servicemix.file.FilePollerEndpoint.processFileNow(FilePollerE ndpoint.java:376) at org.apache.servicemix.file.FilePollerEndpoint$1.run(FilePollerEndpoint.j ava:320) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Regards Guruprasad BH

13 years, 9 months

1
0
0 / 0

Query on JBoss Cache Lock issue on Cluster Environment !!

by Bh, Guruprasad (NSN - IN/Bangalore)

Hi All, I have an issue with the JBoss Cache which we are using as part of Apache Service Mix on a Cluster Environment. We started the service mix process on Cluster environment and tried to trigger some operations such as topology upload which tries to execute an get request operations on the Network Element using the Mediation Framework on a Cluster Environment where it tries to acquire a lock on Cache for each operation triggered when it needs to write some content to Cache. But we observed some issues with this operation trigger which is related to the JBoss Cache. When we triggered the upload operations it tries to acquire a lock on the Cache to write some content but we are getting the exception's as below: 2011-03-17T05:42:42.164+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22544 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000001502, reason :: Unable to acquire lock on Fqn [/jboss/AgentInfo] after [10000] milliseconds for requestor [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22544,5,m ain]]! Lock held by [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,m ain]] 2011-03-17T05:43:43.105+0200 | fifur | ERROR | NE3SWS_MEDIATION_Worker-1 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000003241, reason :: Unable to acquire lock on Fqn [/jboss/AgentInfo] after [10000] milliseconds for requestor [Thread[NE3SWS_MEDIATION_Worker-1,4,main]]! Lock held by [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,] ] 2011-03-17T05:44:21.852+0200 | fifur | ERROR | NE3SWS_MEDIATION_Worker-1 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000003249, reason :: Unable to acquire lock on Fqn [/jboss/AgentInfo] after [10000] milliseconds for requestor [Thread[NE3SWS_MEDIATION_Worker-1,4,main]]! Lock held by [Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541,5,] ] Case1 So if we see the above logs all the new operations triggered are failing to acquire the lock because Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 is holding the lock, So we started checking the status of the thread Thread[pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 and we could never see a thread initiated as part of this process. Case2 Sometimes we were able to find the thread which is holding the lock was terminated with SQL Exception as below 011-03-17T05:44:54.822+0200 | fifur | ERROR | pool-flow.seda.servicemix-scripting-thread-6271 | ScriptingEndpoint | ix.scripting.ScriptingEndpoint 309 | Error executing the script: null at line: -1 and column: -1 javax.script.ScriptException: org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/jboss/AU_STATUS_CACHETYPE] after [10000] milliseconds for requestor [Thread[pool-flow.seda.servicemix-scripting-thread-6271,5,main]]! Lock held by [Thread[pool-flow.seda.servicemix-scripting-thread-6267,5,]] Here we could see that the lock was held by the thread Thread[pool-flow.seda.servicemix-scripting-thread-6267, and we can see from the below logs that the thread was terminated with SQL Exception 2011-03-17T05:41:44.820+0200 | fifur | INFO | pool-flow.seda.servicemix-scripting-thread-6267 | pinger | ect.GeneratedMethodAccessor149 ? | Getting all data from cache 2011-03-17T05:42:32.129+0200 | fifur | WARN | pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 | ManagedConnectionFactory | oader.ManagedConnectionFactory 146 | Failed to close connection java.sql.SQLException: Already closed. at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java :114) at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.clo se(PoolingDataSource.java:191) at org.jboss.cache.loader.ManagedConnectionFactory.safeClose(ManagedConnect ionFactory.java:142) at org.jboss.cache.loader.ManagedConnectionFactory.close(ManagedConnectionF actory.java:129) at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa der.java:427) at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele gatingCacheLoader.java:94) at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo aderInterceptor.java:314) at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand( CacheLoaderInterceptor.java:179) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt erceptor.java:301) at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j ava:283) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo cationContextInterceptor.java:178) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault( InvocationContextInterceptor.java:130) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja va:287) at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati onDelegate.java:494) at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio nDelegate.java:365) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc essor.java:228) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieve(JbossCachePro cessor.java:103) at com.nsn.oss.mf.mediation.ne3sws.ExchangeUtils.retrieveAgentInfo(Exchange Utils.java:177) at com.nsn.oss.mf.mediation.ne3sws.NE3SWSNotificationEndpoint.updateAgentIn fo(NE3SWSNotificationEndpoint.java:460) at com.nsn.oss.mf.mediation.ne3sws.NE3SWSNotificationEndpoint.processInOut( NE3SWSNotificationEndpoint.java:186) at org.apache.servicemix.common.endpoints.ProviderEndpoint.process(Provider Endpoint.java:111) at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC ycle.java:627) at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas eLifeCycle.java:581) at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB aseLifeCycle.java:535) at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync LifeCycleWrapper.java:60) at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D eliveryChannelImpl.java:632) at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j ava:185) at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java :168) at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1 34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 2011-03-17T05:42:32.131+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 | ExchangeUtils | mediation.ne3sws.ExchangeUtils 179 | Error while retrieving the agentInfo null agent unique id :: 6000160000000001502, reason :: java.sql.SQLException: Io exception: Connection reset 2011-03-17T05:42:32.131+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22541 | NE3SWS_MEDIATION | sws.NE3SWSNotificationEndpoint 388 | Fault thrown to agent, message :: Agent not registered fault code :: Client.NotRegistered reason ::Client.NotRegistered 2011-03-17T05:42:36.075+0200 | fifur | WARN | pool-flow.seda.servicemix-scripting-thread-6267 | ManagedConnectionFactory | oader.ManagedConnectionFactory 146 | Failed to close connection java.sql.SQLException: Already closed. at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java :114) at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.clo se(PoolingDataSource.java:191) at org.jboss.cache.loader.ManagedConnectionFactory.safeClose(ManagedConnect ionFactory.java:142) at org.jboss.cache.loader.ManagedConnectionFactory.close(ManagedConnectionF actory.java:129) at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa der.java:427) at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele gatingCacheLoader.java:94) at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo aderInterceptor.java:314) at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand( CacheLoaderInterceptor.java:179) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt erceptor.java:301) at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j ava:283) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo cationContextInterceptor.java:178) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault( InvocationContextInterceptor.java:130) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja va:287) at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati onDelegate.java:494) at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio nDelegate.java:365) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc essor.java:228) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCache Processor.java:122) at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756) at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper .java:766) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.jav a:754) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBy tecodeAdapter.java:170) at Script1.run(Script1.groovy:26) at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24 3) at com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.jav a:46) at javax.script.CompiledScript.eval(Unknown Source) at org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpo int.java:300) at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC ycle.java:627) at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas eLifeCycle.java:581) at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB aseLifeCycle.java:535) at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync LifeCycleWrapper.java:60) at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D eliveryChannelImpl.java:632) at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j ava:185) at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java :168) at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1 34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 2011-03-17T05:42:36.077+0200 | fifur | ERROR | pool-flow.seda.servicemix-scripting-thread-6267 | ScriptingEndpoint | ix.scripting.ScriptingEndpoint 309 | Error executing the script: null at line: -1 and column: -1 javax.script.ScriptException: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24 5) at com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.jav a:46) at javax.script.CompiledScript.eval(Unknown Source) at org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpo int.java:300) at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC ycle.java:627) at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas eLifeCycle.java:581) at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB aseLifeCycle.java:535) at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync LifeCycleWrapper.java:60) at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D eliveryChannelImpl.java:632) at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j ava:185) at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java :168) at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1 34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja va:304) at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati onDelegate.java:494) at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio nDelegate.java:365) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc essor.java:228) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCache Processor.java:122) at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756) at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper .java:766) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.jav a:754) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBy tecodeAdapter.java:170) at Script1.run(Script1.groovy:26) at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24 3) ... 14 more Caused by: java.sql.SQLException: Io exception: Connection reset at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:11 2) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:14 6) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:25 5) at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedSt atement.java:826) at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement. java:1039) at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPrepared Statement.java:850) at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement. java:1134) at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePrepare dStatement.java:3339) at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedSt atement.java:3384) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat ingPreparedStatement.java:96) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat ingPreparedStatement.java:96) at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa der.java:416) at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele gatingCacheLoader.java:94) at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo aderInterceptor.java:314) at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand( CacheLoaderInterceptor.java:179) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt erceptor.java:301) at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j ava:283) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo cationContextInterceptor.java:178) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault( InvocationContextInterceptor.java:130) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja va:287) ... 30 more 2011-03-17T05:42:36.079+0200 | fifur | ERROR | pool-flow.seda.servicemix-scripting-thread-6267 | ScriptingComponent | emix.common.AsyncBaseLifeCycle 537 | Error processing exchange InOnly[ id: ID:10.8.74.54-12ea59a53d1-16:10312 status: Active role: provider service: {http://servicemix.apache.org/test}pinger_script endpoint: endpoint in: <?xml version="1.0" encoding="UTF-8"?><timer><name>{http://servicemix.apache.org/test}au.gen erate_ping.scheduler:endpoint</name><group>DEFAULT</group><fullname>DEFA ULT.{http://servicemix.apache.org/test}au.generate_ping.scheduler:endpoi nt</fullname><description/><fireTime>Thu Mar 17 05:41:44 EET 2011</fireTime></timer> ] javax.script.ScriptException: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24 5) at com.sun.script.groovy.GroovyCompiledScript.eval(GroovyCompiledScript.jav a:46) at javax.script.CompiledScript.eval(Unknown Source) at org.apache.servicemix.scripting.ScriptingEndpoint.process(ScriptingEndpo int.java:300) at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeC ycle.java:627) at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBas eLifeCycle.java:581) at org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncB aseLifeCycle.java:535) at org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(Sync LifeCycleWrapper.java:60) at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(D eliveryChannelImpl.java:632) at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.j ava:185) at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java :168) at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:1 34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.jboss.cache.CacheException: java.sql.SQLException: Io exception: Connection reset at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja va:304) at org.jboss.cache.invocation.CacheInvocationDelegate.getNode(CacheInvocati onDelegate.java:494) at org.jboss.cache.invocation.NodeInvocationDelegate.getChild(NodeInvocatio nDelegate.java:365) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.getNode(JbossCacheProc essor.java:228) at com.nsn.obs.mw.cacheprocessor.JbossCacheProcessor.retrieveAll(JbossCache Processor.java:122) at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756) at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper .java:766) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.jav a:754) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBy tecodeAdapter.java:170) at Script1.run(Script1.groovy:26) at com.sun.script.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:24 3) ... 14 more Caused by: java.sql.SQLException: Io exception: Connection reset at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:11 2) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:14 6) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:25 5) at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedSt atement.java:826) at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement. java:1039) at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPrepared Statement.java:850) at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement. java:1134) at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePrepare dStatement.java:3339) at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedSt atement.java:3384) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat ingPreparedStatement.java:96) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Delegat ingPreparedStatement.java:96) at org.jboss.cache.loader.AdjListJDBCCacheLoader.exists(AdjListJDBCCacheLoa der.java:416) at org.jboss.cache.loader.AbstractDelegatingCacheLoader.exists(AbstractDele gatingCacheLoader.java:94) at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLo aderInterceptor.java:314) at org.jboss.cache.interceptors.CacheLoaderInterceptor.visitGetNodeCommand( CacheLoaderInterceptor.java:179) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInt erceptor.java:301) at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.j ava:283) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(Comma ndInterceptor.java:131) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextIntercept or(CommandInterceptor.java:116) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(Invo cationContextInterceptor.java:178) at org.jboss.cache.interceptors.InvocationContextInterceptor.handleDefault( InvocationContextInterceptor.java:130) at org.jboss.cache.commands.AbstractVisitor.visitGetNodeCommand(AbstractVis itor.java:115) at org.jboss.cache.commands.read.GetNodeCommand.acceptVisitor(GetNodeComman d.java:76) at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.ja va:287) ... 30 more 2011-03-17T05:42:42.164+0200 | fifur | ERROR | pool-flow.seda.ne3sws-mediation-notification-se-thread-22544 | Regards Guruprasad BH

13 years, 9 months

1
0
0 / 0

[JBoss JIRA] Created: (JBCACHE-1598) TreeCache node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found and causing Stuck Thread

by Hans Kristanto (JIRA)

TreeCache node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found and causing Stuck Thread -------------------------------------------------------------------------------------------------------------------- Key: JBCACHE-1598 URL: https://issues.jboss.org/browse/JBCACHE-1598 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Affects Versions: 1.4.1.SP12 Environment: OS : UNIX, Application Server : BEA WebLogic Server 9, Application : Oracle Order and Service Management 6.3.1.53 Reporter: Hans Kristanto Assignee: Manik Surtani When Oracle OSM doing the process, sometimes it throws out infinite loop (a Stuck Thread)like this : <23-Mar-2011 6:44:24,760 GMT+08:00 PM> <WARN> <cache.TreeCache> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <Mar 23, 2011 6:44:24 PM GMT+08:00> <Warning> <oms> <000000> <cache.TreeCache: node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <23-Mar-2011 6:44:24,760 GMT+08:00 PM> <WARN> <cache.TreeCache> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <Mar 23, 2011 6:44:24 PM GMT+08:00> <Warning> <oms> <000000> <cache.TreeCache: node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <23-Mar-2011 6:44:24,760 GMT+08:00 PM> <WARN> <cache.TreeCache> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <Mar 23, 2011 6:44:24 PM GMT+08:00> <Warning> <oms> <000000> <cache.TreeCache: node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <23-Mar-2011 6:44:24,761 GMT+08:00 PM> <WARN> <cache.TreeCache> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <Mar 23, 2011 6:44:24 PM GMT+08:00> <Warning> <oms> <000000> <cache.TreeCache: node /101186/root/0/service/1379/response_automated_tasks/1297826320926 not found> <23-Mar-2011 6:44:24,762 GMT+08:00 PM> <ERROR> <cache.TransactionEntry> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <undo operation failed, error=org.jboss.cache.NodeNotExistsException: node /101186/root/0/service/1379/response_automated_tasks/1295336628571 not found (gtx=GlobalTransaction:<null>:22163, caller=Thread[[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)',1,Pooled Threads])> <Mar 23, 2011 6:44:24 PM GMT+08:00> <Error> <oms> <000000> <cache.TransactionEntry: undo operation failed, error=org.jboss.cache.NodeNotExistsException: node /101186/root/0/service/1379/response_automated_tasks/1295336628571 not found (gtx=GlobalTransaction:<null>:22163, caller=Thread[[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)',1,Pooled Threads])> <23-Mar-2011 6:44:24,762 GMT+08:00 PM> <WARN> <cache.TreeCache> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <node /101186/root/0/service/1379/response_automated_tasks/1295336628571 not found> <Mar 23, 2011 6:44:24 PM GMT+08:00> <Warning> <oms> <000000> <cache.TreeCache: node /101186/root/0/service/1379/response_automated_tasks/1295336628571 not found> I'm not sure wether this issue is already fixed in latest version of JBoss Cache since the code is hide under Oracle OSM Application -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

13 years, 9 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

jbosscache-issues March 2011