[infinispan-issues] [JBoss JIRA] (ISPN-9130) GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC] random failure

[Radim Vansa (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573443#comment-13573443 ] 

Radim Vansa commented on ISPN-9130:
-----------------------------------

I am not sure how to fix this with current design; we could add another instance of ScatteredCacheWriterInterceptor after ScatteredDistributionInterceptor that would write the cache store on primary before returning to SDI, but that's clunky. We could update the persistence directly from SDI but that sounds even worse...

The static nature of the current interceptor stack is rather limiting. I've hit that before, too, in 2LC when distribution interceptor produces a future but we can only wait for that after we unlock the entry. That is solved by returning CF and the locking interceptor knows that if it gets CF, it unlocks & waits for the future to complete, but this breaks the concept of orthogonality/single responsibility. Not that I would have any specific proposal for improvement...



> GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC] random failure
> ---------------------------------------------------------------------------------------------
>
>                 Key: ISPN-9130
>                 URL: https://issues.jboss.org/browse/ISPN-9130
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 9.3.0.Alpha1
>            Reporter: Dan Berindei
>            Assignee: Radim Vansa
>              Labels: testsuite_stability
>             Fix For: 9.3.0.Beta1
>
>         Attachments: GetGroupKeysTest_ISPN-7977_NPE_during_shutdown_20180507.log.gz, ScatteredSyncStoreNotSharedTest_ISPN-7977_NPE_during_shutdown_20180507.log.gz
>
>
> {{ScatteredDistributionInterceptor.handleWriteCommand}} sends the {{PrimaryAckCommand}} to the originator too soon. It takes care of committing the entry to the data container instead of waiting for {{EntryWrappingInterceptor}} to do it, but it doesn't do anything about the stores:
> {noformat}
> 16:02:40,630 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command RemoveCommand{key=GroupKey{group='test-group', key=0}, value=null, metadata=null, flags=[IGNORE_RETURN_VALUES], commandInvocationId=CommandInvocation:Test-NodeB-46322:3254, valueMatcher=MATCH_ALWAYS, topologyId=-1} and InvocationContext [SingleKeyNonTxInvocationContext{isLocked=false, key=null, cacheEntry=null, origin=null, lockOwner=CommandInvocation:Test-NodeB-46322:3254}]
> 16:02:40,631 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [ScatteredDistributionInterceptor] Committing entry RepeatableReadEntry(4631a022){key=GroupKey{group='test-group', key=0}, value=null, isCreated=false, isChanged=true, isRemoved=true, isExpired=false, skipLookup=true, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}}, replaced MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=v0, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=21}}}
> 16:02:40,631 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [DefaultDataContainer] Store MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=null, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}} in container
> 16:02:40,633 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [JGroupsTransport] Test-NodeD-54637 sending command to Test-NodeB-46322: PrimaryAckCommand{id=3254, success=true, value=SimpleClusteredVersion{topologyId=5, version=40}, waitFor=[Test-NodeF-20476]}
> 16:02:40,633 TRACE (jgroups-10,Test-NodeB-46322:[]) [DefaultDataContainer] Store MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=null, metadata=EmbeddedMetadata{version=SimpleClusteredVersion{topologyId=5, version=40}}} in container
> 16:02:40,634 TRACE (jgroups-10,Test-NodeB-46322:[]) [DistCacheWriterInterceptor] Stored entry null under key GroupKey{group='test-group', key=0}
> ---
> 16:02:40,634 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command GetKeysInGroupCommand{groupName='test-group', flags=[]} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext at 6314f4e7]
> 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [InvocationContextInterceptor] Invoked with command GetKeysInGroupCommand{groupName='test-group', flags=[]} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext at 6dd409f2]
> 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [DummyInMemoryStore] Processing entries in store null with filter org.infinispan.filter.CompositeKeyFilter at 196d8e49 and callback org.infinispan.interceptors.impl.CacheLoaderInterceptor$1 at 54df66d9
> 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [EntryFactoryImpl] Creating new entry for key GroupKey{group='test-group', key=0}
> 16:02:40,636 TRACE (jgroups-4,Test-NodeD-54637:[]) [JGroupsTransport] Test-NodeD-54637 sending response for request 21 to Test-NodeB-46322: SuccessfulResponse([MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=v0, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=21}}}])
> ---
> 16:02:40,636 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [DummyInMemoryStore] Store MarshalledEntryImpl{keyBytes=null, valueBytes=null, metadataBytes=null, key=GroupKey{group='test-group', key=0}, value=null, metadata=InternalMetadataImpl{actual=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}, created=-1, lastUsed=-1}, marshaller=org.infinispan.marshall.core.GlobalMarshaller at 26ecc0d1} in dummy map store at 5d185974
> ---
> 16:02:40,638 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC]
> java.lang.AssertionError: 
> 	at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.9.9.jar:?]
> 	at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24) ~[testng-6.9.9.jar:?]
> 	at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33) ~[testng-6.9.9.jar:?]
> 	at org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistence(GetGroupKeysTest.java:162) ~[test-classes/:?]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.5.0#75005)