DistSyncL1FuncTest.testEntryInL1ReplaceWithConcurrentPut random failures ------------------------------------------------------------------------ Key: ISPN-3760 URL: https://issues.jboss.org/browse/ISPN-3760 Project: Infinispan Issue Type: Bug Components: Test Suite - Core Affects Versions: 6.0.0.Final Reporter: Dan Berindei Assignee: William Burns Labels: testsuite_stability Fix For: 7.0.0.Final The test does a {{put(k, v)}} from an owner, then a {{get(k)}} from a non-owner, and assumes that {{k}} is now in the non-owner's L1 cache. But a previous test, {{testEntryInL1ReplaceWithConcurrentInvalidation}}, added the non-owner as a L1 requestor on the backup owner, which will try to invalidate the key on the non-owner. Because the L1 invalidation is asynchronous, it can reach the non-owner after the get was issued and invalidate the L1 entry: {noformat} 12:23:57,332 TRACE (testng-DistSyncL1FuncTest:dist) [InvocationContextInterceptor] Invoked with command PutKeyValueCommand{key=key-to-the-cache, value=first-put, flags=null, putIfAbsent=false, valueMatchingPolicy=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@5caff826] 12:23:57,332 TRACE (testng-DistSyncL1FuncTest:dist) [JGroupsTransport] dests=[NodeD-17616, NodeC-12131], command=SingleRpcCommand{cacheName='dist', command=PutKeyValueCommand{key=key-to-the-cache, value=first-put, flags=null, putIfAbsent=false, valueMatchingPolicy=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}}, mode=SYNCHRONOUS, timeout=60000 12:23:57,334 TRACE (asyncTransportThread-0,NodeC:) [JGroupsTransport] dests=[NodeA-23135], command=SingleRpcCommand{cacheName='dist', command=InvalidateL1Command{num keys=1, forRehash=false, origin=NodeD-17616}}, mode=SYNCHRONOUS, timeout=60000 12:23:57,339 TRACE (remote-thread-0,NodeC:dist) [DefaultDataContainer] Store ImmortalCacheEntry{key=key-to-the-cache, value=first-put} in container 12:23:57,339 TRACE (testng-DistSyncL1FuncTest:dist) [DefaultDataContainer] Store ImmortalCacheEntry{key=key-to-the-cache, value=first-put} in container 12:23:57,339 TRACE (testng-DistSyncL1FuncTest:dist) [L1ManagerImpl] Invalidating keys [key-to-the-cache] on nodes [NodeA-23135]. Use multicast? false 12:23:57,340 TRACE (testng-DistSyncL1FuncTest:dist) [InvocationContextInterceptor] Invoked with command GetKeyValueCommand {key=key-to-the-cache, flags=null} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@359e1aed] 12:23:57,340 TRACE (asyncTransportThread-4,NodeD:) [JGroupsTransport] dests=[NodeA-23135], command=SingleRpcCommand{cacheName='dist', command=InvalidateL1Command{num keys=1, forRehash=false, origin=null}}, mode=SYNCHRONOUS, timeout=60000 12:23:57,340 TRACE (testng-DistSyncL1FuncTest:dist) [JGroupsTransport] dests=[NodeD-17616, NodeC-12131], command=ClusteredGetCommand{key=key-to-the-cache, flags=null}, mode=WAIT_FOR_VALID_RESPONSE, timeout=60000 12:23:57,340 TRACE (remote-thread-1,NodeA:dist) [L1NonTxInterceptor] Aborted possible L1 update due to concurrent invalidation for key key-to-the-cache 12:23:57,341 ERROR (testng-DistSyncL1FuncTest:) [UnitTestTestNGListener] Test testEntryInL1ReplaceWithConcurrentPut(org.infinispan.distribution.DistSyncL1FuncTest) failed. java.lang.AssertionError: Entry for key [key-to-the-cache] should be in L1 on cache at [NodeA-23135]! at org.infinispan.distribution.DistributionTestHelper.assertIsInL1(DistributionTestHelper.java:31) at org.infinispan.distribution.BaseDistFunctionalTest.assertIsInL1(BaseDistFunctionalTest.java:183) at org.infinispan.distribution.DistSyncL1FuncTest.testEntryInL1ReplaceWithConcurrentPut(DistSyncL1FuncTest.java:180) {noformat}