GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC] random failure --------------------------------------------------------------------------------------------- Key: ISPN-9130 URL: https://issues.jboss.org/browse/ISPN-9130 Project: Infinispan Issue Type: Bug Components: Core Affects Versions: 9.3.0.Alpha1 Reporter: Dan Berindei Assignee: Radim Vansa Labels: testsuite_stability Fix For: 9.3.0.CR1 Attachments: GetGroupKeysTest_ISPN-7977_NPE_during_shutdown_20180507.log.gz, ScatteredSyncStoreNotSharedTest_ISPN-7977_NPE_during_shutdown_20180507.log.gz {{ScatteredDistributionInterceptor.handleWriteCommand}} sends the {{PrimaryAckCommand}} to the originator too soon. It takes care of committing the entry to the data container instead of waiting for {{EntryWrappingInterceptor}} to do it, but it doesn't do anything about the stores: {noformat} 16:02:40,630 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command RemoveCommand{key=GroupKey{group='test-group', key=0}, value=null, metadata=null, flags=[IGNORE_RETURN_VALUES], commandInvocationId=CommandInvocation:Test-NodeB-46322:3254, valueMatcher=MATCH_ALWAYS, topologyId=-1} and InvocationContext [SingleKeyNonTxInvocationContext{isLocked=false, key=null, cacheEntry=null, origin=null, lockOwner=CommandInvocation:Test-NodeB-46322:3254}] 16:02:40,631 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [ScatteredDistributionInterceptor] Committing entry RepeatableReadEntry(4631a022){key=GroupKey{group='test-group', key=0}, value=null, isCreated=false, isChanged=true, isRemoved=true, isExpired=false, skipLookup=true, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}}, replaced MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=v0, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=21}}} 16:02:40,631 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [DefaultDataContainer] Store MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=null, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}} in container 16:02:40,633 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [JGroupsTransport] Test-NodeD-54637 sending command to Test-NodeB-46322: PrimaryAckCommand{id=3254, success=true, value=SimpleClusteredVersion{topologyId=5, version=40}, waitFor=[Test-NodeF-20476]} 16:02:40,633 TRACE (jgroups-10,Test-NodeB-46322:[]) [DefaultDataContainer] Store MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=null, metadata=EmbeddedMetadata{version=SimpleClusteredVersion{topologyId=5, version=40}}} in container 16:02:40,634 TRACE (jgroups-10,Test-NodeB-46322:[]) [DistCacheWriterInterceptor] Stored entry null under key GroupKey{group='test-group', key=0} --- 16:02:40,634 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command GetKeysInGroupCommand{groupName='test-group', flags=[]} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@6314f4e7] 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [InvocationContextInterceptor] Invoked with command GetKeysInGroupCommand{groupName='test-group', flags=[]} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@6dd409f2] 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [DummyInMemoryStore] Processing entries in store null with filter org.infinispan.filter.CompositeKeyFilter@196d8e49 and callback org.infinispan.interceptors.impl.CacheLoaderInterceptor$1@54df66d9 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [EntryFactoryImpl] Creating new entry for key GroupKey{group='test-group', key=0} 16:02:40,636 TRACE (jgroups-4,Test-NodeD-54637:[]) [JGroupsTransport] Test-NodeD-54637 sending response for request 21 to Test-NodeB-46322: SuccessfulResponse([MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=v0, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=21}}}]) --- 16:02:40,636 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [DummyInMemoryStore] Store MarshalledEntryImpl{keyBytes=null, valueBytes=null, metadataBytes=null, key=GroupKey{group='test-group', key=0}, value=null, metadata=InternalMetadataImpl{actual=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}, created=-1, lastUsed=-1}, marshaller=org.infinispan.marshall.core.GlobalMarshaller@26ecc0d1} in dummy map store@5d185974 --- 16:02:40,638 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC] java.lang.AssertionError: at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.9.9.jar:?] at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24) ~[testng-6.9.9.jar:?] at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33) ~[testng-6.9.9.jar:?] at org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistence(GetGroupKeysTest.java:162) ~[test-classes/:?] {noformat}