[jbosscache-issues] [JBoss JIRA] Commented: (JBCACHE-1565) Locking doesn't work when replication is used

[Lillian Andres (JIRA)]

    [ https://jira.jboss.org/jira/browse/JBCACHE-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12525214#action_12525214 ] 

Lillian Andres commented on JBCACHE-1565:
-----------------------------------------

Our team is also experiencing the same problem when we have the unfortunate case of multiple writers on the same exact pojo object.  A partial stack dump follows.

Unfortunately, the problem is a show stopper for us.  Is there any workaround?  
1.  Can we use the a different locking scheme besides MVCC?  
2.  Can we catch the timeout exception and totally remove the node and re- add it?  Is there a way where we can via software program release the lock?

Thank you for your help.

Caused by: org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/testcache/Test] after [60000] milliseconds for requestor [GlobalTransaction:<127.0.0.0:2103>:294]! Lock held by [GlobalTransaction:<127.0.0.0:2103>:274]
                at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:170)
                at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:226)
                at org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:98)
                at org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
                at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
                at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
                at org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
                at org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
                at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
                at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
                at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
                at org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
                at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
                at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
                at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:284)
                at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:271)
                at org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
                at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
                at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
                at org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:119)
                at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
                at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
                at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
                at org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutKeyValueCommand(InvocationContextInterceptor.java:82)
                at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
                at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:265)
                at org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:560)
                at org.jboss.cache.pojo.impl.InternalHelper.lockPojo(InternalHelper.java:342)
                at org.jboss.cache.pojo.impl.PojoCacheDelegate.putObject(PojoCacheDelegate.java:101)
                at org.jboss.cache.pojo.impl.PojoCacheImpl.attach(PojoCacheImpl.java:102)
                ... 43 more


> Locking doesn't work when replication is used
> ---------------------------------------------
>
>                 Key: JBCACHE-1565
>                 URL: https://jira.jboss.org/jira/browse/JBCACHE-1565
>             Project: JBoss Cache
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>    Affects Versions: 3.2.1.GA
>         Environment: Linux, Sun Java 5.0
>            Reporter: tbech
>            Assignee: Mircea Markus
>            Priority: Critical
>         Attachments: ConcurrentWritersTest.java, exc.txt, GetPutCachebenchCluster.xml, GetPutTest.java, JBossCache321Wrapper.java, mvcc-repl-sync.xml
>
>
> During performance tests of JBossCache, I've tried to test transactions with  cache.getInvocationContext().getOptionOverrides().setForceWriteLock operation ('select-for-update'). I've used standard jbosscache benchmark from svn. I've created new product jbosscache3.2.1 based on 3.1.0 and new test. Basicaly it did:
>          try
>          {
>             long startTime = System.nanoTime();
>             if (concurrentWrites) {
> 	            cacheWrapper.startTransaction();
> 	            cacheWrapper.setForceWriteLock(true);
>             }
>             ObjectInCache o = (ObjectInCache) cacheWrapper.get(path, key);
>             o = new ObjectInCache(Integer.valueOf(o.getVal().intValue() + 1), randStr[taskNumber%randStr.length]);
>             cacheWrapper.put(path, key, o);
>             if (concurrentWrites) {
>             	cacheWrapper.endTransaction(true);
>             }
>             long statValue = (System.nanoTime() - startTime);
>             descriptiveStatistics.addValue(statValue);
>          }
>          catch (Throwable e)
>          {
>             log.error("Error appeared whilst writing to cache:" + e.getMessage(), e);
>             try {
>                 if (concurrentWrites) {
>                 	cacheWrapper.endTransaction(false);
>                 }
>             } finally {
>             	
>             }
>          } finally {
>             logRunCount(taskNumber);
>          }
>  concurrentWrites was set to true.
> This tests passes nicely in one node (local mode). However in 2,3,4 nodes it fails after first usage of the same object by different node with exception:
> org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/concurrent/Intermediate-0/Node 0] after [240000] milliseconds for requestor [GlobalTransaction:<10.118.236.212:40325>:0]! Lock held by [GlobalTransaction:<10.118.236.211:8112>:40]
>         at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
> It looks like, in replication the node is not beeing releases after transaction commit.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira