[JBoss JIRA] (ISPN-4384) CacheNotifierImplInitialTransferDistTest test instability
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4384?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4384:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1148564|https://bugzilla.redhat.com/show_bug.cgi?id=1148564] from VERIFIED to CLOSED
> CacheNotifierImplInitialTransferDistTest test instability
> ---------------------------------------------------------
>
> Key: ISPN-4384
> URL: https://issues.jboss.org/browse/ISPN-4384
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 7.0.0.Alpha5
>
>
> There was an issue that was just recently fixed for
> However there are some other inconsistent failures as seen here
> http://ci.infinispan.org/viewLog.html?buildId=8820&tab=buildResultsDiv&bu...
> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered
> {code}
> java.lang.AssertionError: expected [11] but found [6]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertEquals(Assert.java:123)
> at org.testng.Assert.assertEquals(Assert.java:370)
> at org.testng.Assert.assertEquals(Assert.java:380)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:510)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:600)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> {code}
> and CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered has the same stack trace.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years, 2 months
[JBoss JIRA] (ISPN-4370) CacheNotifierImplInitialTransferDistTest random failures
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4370?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4370:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1148564|https://bugzilla.redhat.com/show_bug.cgi?id=1148564] from VERIFIED to CLOSED
> CacheNotifierImplInitialTransferDistTest random failures
> --------------------------------------------------------
>
> Key: ISPN-4370
> URL: https://issues.jboss.org/browse/ISPN-4370
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: William Burns
> Labels: testsuite_stability
> Fix For: 7.0.0.Alpha5
>
> Attachments: CacheNotifierImplInitialTransferDistTest_master_20140612.log.gz, CacheNotifierImplInitialTransferDistTest_t_ISPN-4118_20140609.log.gz
>
>
> Random failures in {{testModificationAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered}} and {{testRemoveAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered}}.
> {noformat}
> java.lang.AssertionError: There was no matching create event for key key-1 expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:542)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testRemoveAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:655)
> {noformat}
> {noformat}
> java.lang.AssertionError: There was no matching create event for key key-1 expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:542)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testRemoveAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:655)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years, 2 months
[JBoss JIRA] (ISPN-4384) CacheNotifierImplInitialTransferDistTest test instability
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4384?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4384:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1148564|https://bugzilla.redhat.com/show_bug.cgi?id=1148564] from VERIFIED to CLOSED
> CacheNotifierImplInitialTransferDistTest test instability
> ---------------------------------------------------------
>
> Key: ISPN-4384
> URL: https://issues.jboss.org/browse/ISPN-4384
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 7.0.0.Alpha5
>
>
> There was an issue that was just recently fixed for
> However there are some other inconsistent failures as seen here
> http://ci.infinispan.org/viewLog.html?buildId=8820&tab=buildResultsDiv&bu...
> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered
> {code}
> java.lang.AssertionError: expected [11] but found [6]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertEquals(Assert.java:123)
> at org.testng.Assert.assertEquals(Assert.java:370)
> at org.testng.Assert.assertEquals(Assert.java:380)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:510)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueNonOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:600)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> {code}
> and CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered has the same stack trace.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years, 2 months
[JBoss JIRA] (ISPN-4370) CacheNotifierImplInitialTransferDistTest random failures
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4370?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4370:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1148564|https://bugzilla.redhat.com/show_bug.cgi?id=1148564] from VERIFIED to CLOSED
> CacheNotifierImplInitialTransferDistTest random failures
> --------------------------------------------------------
>
> Key: ISPN-4370
> URL: https://issues.jboss.org/browse/ISPN-4370
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: William Burns
> Labels: testsuite_stability
> Fix For: 7.0.0.Alpha5
>
> Attachments: CacheNotifierImplInitialTransferDistTest_master_20140612.log.gz, CacheNotifierImplInitialTransferDistTest_t_ISPN-4118_20140609.log.gz
>
>
> Random failures in {{testModificationAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered}} and {{testRemoveAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered}}.
> {noformat}
> java.lang.AssertionError: There was no matching create event for key key-1 expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:542)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testRemoveAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:655)
> {noformat}
> {noformat}
> java.lang.AssertionError: There was no matching create event for key key-1 expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testIterationBeganAndSegmentNotComplete(CacheNotifierImplInitialTransferDistTest.java:542)
> at org.infinispan.notifications.cachelistener.CacheNotifierImplInitialTransferDistTest.testRemoveAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered(CacheNotifierImplInitialTransferDistTest.java:655)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years, 2 months
[JBoss JIRA] (ISPN-4602) Verify EntryIterator works with MarshalledValues
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4602?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4602:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1139629|https://bugzilla.redhat.com/show_bug.cgi?id=1139629] from VERIFIED to CLOSED
> Verify EntryIterator works with MarshalledValues
> ------------------------------------------------
>
> Key: ISPN-4602
> URL: https://issues.jboss.org/browse/ISPN-4602
> Project: Infinispan
> Issue Type: Bug
> Components: Marshalling
> Affects Versions: 7.0.0.Alpha5
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 7.0.0.Beta1
>
>
> The EntryIterator currently doesn't deserialize MarshalledValues as needed which would cause filter failures and the incorrect values to be returned.
> This also means each key/value pair would need to be deserialized when applied to filter which will be slower and should be noted in documentation, but sent across as MarshalledValues?. The only other way is to use some sort of proxy for each object to force lazy deserialization on referencing a field when applying filter, but this seems overkill.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years, 2 months
[JBoss JIRA] (ISPN-4872) During state transfer, a pessimistic transaction can fail due to a ClusteredGetCommand forcing a lock command
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4872?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4872:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1160637|https://bugzilla.redhat.com/show_bug.cgi?id=1160637] from VERIFIED to CLOSED
> During state transfer, a pessimistic transaction can fail due to a ClusteredGetCommand forcing a lock command
> -------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-4872
> URL: https://issues.jboss.org/browse/ISPN-4872
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 5.2.7.Final, 6.0.2.Final, 7.0.0.CR2
> Reporter: Erik Salter
> Assignee: Dan Berindei
> Fix For: 7.0.0.Final
>
>
> This is a strange one:
> A pessimistic transaction started on the primary owner during state transfer can fail because a backup owner issues a ClusteredGetCommand to the primary while processing the write command. This can happen if the backup needs to perform a remote get (due to a DELTA_WRITE, conditional, or reliable return values). In this case, it's a DELTA_WRITE.
> In this chain of events, state transfer is ongoing, and the union CH is (east-dht5, west-dht5, east-dht6) In this case, the pessimistic transaction originated on east-dht5 must be invoked on west-dht5 and east-dht6:
> {code}
> 2014-10-21 02:15:01,309 TRACE [org.infinispan.transaction.TransactionTable] (OOB-1713,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Created a new local transaction: LocalXaTransaction{xid=null} LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, isFromStateTransfer=false} globalTx=GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local, topologyId=33, age(ms)=0
> 2014-10-21 02:15:01,309 TRACE [org.infinispan.util.concurrent.locks.containers.OwnableReentrantPerEntryLockContainer] (OOB-1713,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Creating and acquiring new lock instance for key EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831]
> 2014-10-21 02:15:01,345 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-1713,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Replication task sending PrepareCommand {modifications=[PutKeyValueCommand{key=EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831], value=QamResource [qamId=431831, currentBandwidth=3750000, maxBandwidth=38810000, maxPrograms=255, currentPrograms=1], flags=[SKIP_REMOTE_LOOKUP, DELTA_WRITE], putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1, successful=true, ignorePreviousValue=false}], onePhaseCommit=true, gtx=GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local, cacheName='qamAllocation', topologyId=33} to addresses [240-west-dht5.comcast.net-35898(CH2-Chicago-IL), 240-east-dht6.comcast.net-47049(CMC-Denver-CO)] with response mode GET_ALL
> {code}
> Let's examine east-dht6. It gets the write request. It needs to perform a remote get to pull the full key context, since it receives only the delta.
> {code}
> 2014-10-21 02:15:01,362 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-2149,session-resource-cluster,240-east-dht6.comcast.net-47049(CMC-Denver-CO)) Attempting to execute command: PrepareCommand {modifications=[PutKeyValueCommand{key=EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831], value=QamResourceDelta [qamId=431831, basePort=2049, currentBandwidth=3750000, maxBandwidth=38810000, maxPrograms=255, currentPrograms=1, baseProgramNumber=1, portStep=1, program=Program [portNumber=2053, programNumber=5, status=UNUSED, sessionToken=, lastUpdatedTime=1413872101310, bandwidth=0, inputId=-1]], flags=[SKIP_REMOTE_LOOKUP, DELTA_WRITE], putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1, successful=true, ignorePreviousValue=false}], onePhaseCommit=true, gtx=GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local, cacheName='qamAllocation', topologyId=33} [sender=240-east-dht5.comcast.net-26106(CMC-Denver-CO)]
> ...
> 2014-10-21 02:15:01,362 TRACE [org.infinispan.transaction.TransactionTable] (OOB-2149,session-resource-cluster,240-east-dht6.comcast.net-47049(CMC-Denver-CO)) Created and registered remote transaction RemoteTransaction{modifications=...
> globalTx=GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:remote, topologyId=33, age(ms)=0
> 2014-10-21 02:15:01,362 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-2149,session-resource-cluster,240-east-dht6.comcast.net-47049(CMC-Denver-CO)) Replication task sending ClusteredGetCommand{key=EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831], flags=[SKIP_REMOTE_LOOKUP, DELTA_WRITE]} to addresses [240-east-dht5.comcast.net-26106(CMC-Denver-CO), 240-west-dht5.comcast.net-35898(CH2-Chicago-IL)] with response mode GET_FIRST
> {code}
> east-dht5 receives the ClusteredGetCommand request. Because TxDistributionInterceptor::line 325 is true, the acquireRemoteLock flag is set to true.
> {code}
> boolean acquireRemoteLock = false;
> if (ctx.isInTxScope()) {
> TxInvocationContext txContext = (TxInvocationContext) ctx;
> acquireRemoteLock = isWrite && isPessimisticCache && !txContext.getAffectedKeys().contains(key);
> }
> {code}
> Thus, when east-dht5 runs the ClusteredGetCommand. it will create and invoke a LockControlCommand. Because the LCC is created inline, it will not have its origin set. But the LCC will create a RemoteTxInvocationContext. When the LCC runs through the interceptor chain, TxInterceptor::invokeNextInterceptorAndVerifyTransaction will check the originator (because of the remote context). When it doesn't find it, it will force a rollback.
> {code}
> 2014-10-21 02:15:01,347 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (OOB-1721,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Calling perform() on ClusteredGetCommand{key=EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831], flags=[SKIP_REMOTE_LOOKUP, DELTA_WRITE]}
> 2014-10-21 02:15:01,347 TRACE [org.infinispan.transaction.TransactionTable] (OOB-1721,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Created and registered remote transaction RemoteTransaction{modifications=[], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=null, missingLookedUpEntries=false, isMarkedForRollback=false} globalTx=GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local, topologyId=33, age(ms)=0
> 2014-10-21 02:15:01,347 TRACE [org.infinispan.statetransfer.StateTransferInterceptor] (OOB-1721,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) handleTopologyAffectedCommand for command LockControlCommand{cache=qamAllocation, keys=[EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831]], flags=[SKIP_REMOTE_LOOKUP, DELTA_WRITE], unlock=false}
> 2014-10-21 02:15:01,347 TRACE [org.infinispan.interceptors.locking.PessimisticLockingInterceptor] (OOB-1721,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Locking key EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831], no need to check for pending locks.
> 2014-10-21 02:15:01,347 TRACE [org.infinispan.interceptors.TxInterceptor] (OOB-1721,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) invokeNextInterceptorAndVerifyTransaction :: originatorMissing=true, alreadyCompleted=false
> 2014-10-21 02:15:01,347 TRACE [org.infinispan.interceptors.TxInterceptor] (OOB-1721,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) Rolling back remote transaction GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local because either already completed(false) or originator(null) no longer in the cluster(true).
>
> // ROLLBACK HAPPENS HERE
> {code}
> When the transaction attempts to commit, it fails due to the spurious rollback. For instance:
> {code}
> 2014-10-21 02:15:01,384 WARN [org.infinispan.remoting.rpc.RpcManagerImpl] (OOB-1713,session-resource-cluster,240-east-dht5.comcast.net-26106(CMC-Denver-CO)) ISPN000071: Caught exception when handling command PrepareCommand {modifications=[PutKeyValueCommand{key=EdgeResourceCacheKey[edgeDeviceId=4211,resourceId=431831], value=QamResource [qamId=431831, currentBandwidth=3750000, maxBandwidth=38810000, maxPrograms=255, currentPrograms=1], flags=[SKIP_REMOTE_LOOKUP, DELTA_WRITE], putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1, successful=true, ignorePreviousValue=false}], onePhaseCommit=true, gtx=GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local, cacheName='qamAllocation', topologyId=33}
> org.infinispan.remoting.RemoteException: ISPN000217: Received exception from 240-west-dht5.comcast.net-35898(CH2-Chicago-IL), see cause for remote stack trace
> ...
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:175)
> at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:122)
> ...
> com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:636)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2619)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1779)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:88)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
> Caused by: org.infinispan.transaction.xa.InvalidTransactionException: This remote transaction GlobalTransaction:<240-east-dht5.comcast.net-26106(CMC-Denver-CO)>:1575124:local is already rolled back
> at org.infinispan.transaction.RemoteTransaction.checkIfRolledBack(RemoteTransaction.java:137)
> at org.infinispan.transaction.RemoteTransaction.putLookedUpEntry(RemoteTransaction.java:73)
> at org.infinispan.context.impl.RemoteTxInvocationContext.putLookedUpEntry(RemoteTxInvocationContext.java:99)
> at org.infinispan.container.EntryFactoryImpl.wrapInternalCacheEntryForPut(EntryFactoryImpl.java:242)
> at org.infinispan.container.EntryFactoryImpl.wrapEntryForPut(EntryFactoryImpl.java:166)
> ...
> {code}
> So it looks like there's a couple ways to handle this. One would be to only acquire a remote lock in TxDistributionInterceptor if the current node was the originator. Another -- possibly less intrusive -- would be to set the origin of the invoked LCC command to that of the ClusteredGetCommand. The former should be preferable, but this code tends to be a bit labyrinthine with the various pessimistic use cases out there.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years, 2 months