[JBoss JIRA] (ISPN-9178) EntityCollectionInvalidationTest.testAll randomly failing
by Vittorio Rigamonti (JIRA)
[ https://issues.jboss.org/browse/ISPN-9178?page=com.atlassian.jira.plugin.... ]
Vittorio Rigamonti updated ISPN-9178:
-------------------------------------
Fix Version/s: (was: 9.3.0.CR1)
> EntityCollectionInvalidationTest.testAll randomly failing
> ---------------------------------------------------------
>
> Key: ISPN-9178
> URL: https://issues.jboss.org/browse/ISPN-9178
> Project: Infinispan
> Issue Type: Bug
> Components: Hibernate Cache
> Affects Versions: 9.3.0.Beta1
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Labels: testsuite_stability
> Fix For: 9.3.0.Final
>
> Attachments: ispn9178-log-v2.tgz
>
>
> Failure:
> {code}
> [INFO] Running
> org.infinispan.test.hibernate.cache.commons.functional.cluster.EntityCollectionInvalidationTest
> [ERROR] Tests run: 12, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 7.655 s <<< FAILURE! - in org.infinispan.test.hibernate.cache.commons.functional.cluster.EntityCollectionInvalidationTest
> [ERROR] testAll[nonstrict, REPL_SYNC](org.infinispan.test.hibernate.cache.commons.functional.cluster.EntityCollectionInvalidationTest) Time elapsed: 0.563 s <<< FAILURE!
> java.lang.AssertionError: Contact#1 was in cache
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.infinispan.test.hibernate.cache.commons.functional.cluster.EntityCollectionInvalidationTest.assertLoadedFromCache(EntityCollectionInvalidationTest.java:426)
> at org.infinispan.test.hibernate.cache.commons.functional.cluster.EntityCollectionInvalidationTest.testAll(EntityCollectionInvalidationTest.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at org.hibernate.testing.junit4.ExtendedFrameworkMethod.invokeExplosively(ExtendedFrameworkMethod.java:45)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> TRACE logs attached.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 10 months
[JBoss JIRA] (ISPN-9175) Mark Hibernate 2LC tests unstable
by Vittorio Rigamonti (JIRA)
[ https://issues.jboss.org/browse/ISPN-9175?page=com.atlassian.jira.plugin.... ]
Vittorio Rigamonti updated ISPN-9175:
-------------------------------------
Fix Version/s: 9.3.0.Final
(was: 9.3.0.CR1)
> Mark Hibernate 2LC tests unstable
> ---------------------------------
>
> Key: ISPN-9175
> URL: https://issues.jboss.org/browse/ISPN-9175
> Project: Infinispan
> Issue Type: Task
> Components: Hibernate Cache
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 9.3.0.Final
>
>
> Recent known failures:
> 1. Spotted in PR for ISPN-8601 but probably not related to that JIRA:
> {code}
> EntityRegionAccessStrategyTest.testUpdate[JTA, REPL_SYNC,AccessType[read-write]]
> EntityRegionAccessStrategyTest.testRemoveAll[JTA, REPL_SYNC, AccessType[read-only]]
> {code}
> 2. Unclear when the first time it happened:
> {code}EntityCollectionInvalidationTest.testAll[read-write, DIST_SYNC]{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 10 months
[JBoss JIRA] (ISPN-9173) Availability mode should be updated atomically with the actual members
by Vittorio Rigamonti (JIRA)
[ https://issues.jboss.org/browse/ISPN-9173?page=com.atlassian.jira.plugin.... ]
Vittorio Rigamonti updated ISPN-9173:
-------------------------------------
Fix Version/s: 9.3.0.Final
(was: 9.3.0.CR1)
> Availability mode should be updated atomically with the actual members
> ----------------------------------------------------------------------
>
> Key: ISPN-9173
> URL: https://issues.jboss.org/browse/ISPN-9173
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.3.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.3.0.Final
>
>
> This is a follow-up on ISPN-7682, which asks for the topology itself to be updated atomically.
> {{LocalTopologyManagerImpl}} has additional logic to update the availability mode first when the cache becomes degraded and to update it last when the cache becomes available, which means any delay between the updates cannot cause data inconsistencies.
> But that logic doesn't really belong in {{LocalTopologyManagerImpl}}, and it's easy to forget it's there (and in fact we had a bug there related to the new rebalance phases).
> In addition, tests that want to check the cache behaviour in degraded mode and wait only for the availability mode change will fail if there's a big delay between the availability mode change. I actually hit this while testing my ISPN-8731/ISPN-7682 changes, and I had added a random delay in {{StateConsumerImpl}} before {{distributionManager.setCacheTopology()}}.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 10 months
[JBoss JIRA] (ISPN-9130) GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC] random failure
by Vittorio Rigamonti (JIRA)
[ https://issues.jboss.org/browse/ISPN-9130?page=com.atlassian.jira.plugin.... ]
Vittorio Rigamonti updated ISPN-9130:
-------------------------------------
Fix Version/s: 9.3.0.Final
(was: 9.3.0.CR1)
> GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC] random failure
> ---------------------------------------------------------------------------------------------
>
> Key: ISPN-9130
> URL: https://issues.jboss.org/browse/ISPN-9130
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.3.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Radim Vansa
> Labels: testsuite_stability
> Fix For: 9.3.0.Final
>
> Attachments: GetGroupKeysTest_ISPN-7977_NPE_during_shutdown_20180507.log.gz, ScatteredSyncStoreNotSharedTest_ISPN-7977_NPE_during_shutdown_20180507.log.gz
>
>
> {{ScatteredDistributionInterceptor.handleWriteCommand}} sends the {{PrimaryAckCommand}} to the originator too soon. It takes care of committing the entry to the data container instead of waiting for {{EntryWrappingInterceptor}} to do it, but it doesn't do anything about the stores:
> {noformat}
> 16:02:40,630 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command RemoveCommand{key=GroupKey{group='test-group', key=0}, value=null, metadata=null, flags=[IGNORE_RETURN_VALUES], commandInvocationId=CommandInvocation:Test-NodeB-46322:3254, valueMatcher=MATCH_ALWAYS, topologyId=-1} and InvocationContext [SingleKeyNonTxInvocationContext{isLocked=false, key=null, cacheEntry=null, origin=null, lockOwner=CommandInvocation:Test-NodeB-46322:3254}]
> 16:02:40,631 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [ScatteredDistributionInterceptor] Committing entry RepeatableReadEntry(4631a022){key=GroupKey{group='test-group', key=0}, value=null, isCreated=false, isChanged=true, isRemoved=true, isExpired=false, skipLookup=true, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}}, replaced MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=v0, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=21}}}
> 16:02:40,631 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [DefaultDataContainer] Store MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=null, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}} in container
> 16:02:40,633 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [JGroupsTransport] Test-NodeD-54637 sending command to Test-NodeB-46322: PrimaryAckCommand{id=3254, success=true, value=SimpleClusteredVersion{topologyId=5, version=40}, waitFor=[Test-NodeF-20476]}
> 16:02:40,633 TRACE (jgroups-10,Test-NodeB-46322:[]) [DefaultDataContainer] Store MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=null, metadata=EmbeddedMetadata{version=SimpleClusteredVersion{topologyId=5, version=40}}} in container
> 16:02:40,634 TRACE (jgroups-10,Test-NodeB-46322:[]) [DistCacheWriterInterceptor] Stored entry null under key GroupKey{group='test-group', key=0}
> ---
> 16:02:40,634 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command GetKeysInGroupCommand{groupName='test-group', flags=[]} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@6314f4e7]
> 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [InvocationContextInterceptor] Invoked with command GetKeysInGroupCommand{groupName='test-group', flags=[]} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@6dd409f2]
> 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [DummyInMemoryStore] Processing entries in store null with filter org.infinispan.filter.CompositeKeyFilter@196d8e49 and callback org.infinispan.interceptors.impl.CacheLoaderInterceptor$1@54df66d9
> 16:02:40,635 TRACE (jgroups-4,Test-NodeD-54637:[]) [EntryFactoryImpl] Creating new entry for key GroupKey{group='test-group', key=0}
> 16:02:40,636 TRACE (jgroups-4,Test-NodeD-54637:[]) [JGroupsTransport] Test-NodeD-54637 sending response for request 21 to Test-NodeB-46322: SuccessfulResponse([MetadataImmortalCacheEntry{key=GroupKey{group='test-group', key=0}, value=v0, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=21}}}])
> ---
> 16:02:40,636 TRACE (remote-thread-Test-NodeD-p5380-t5:[]) [DummyInMemoryStore] Store MarshalledEntryImpl{keyBytes=null, valueBytes=null, metadataBytes=null, key=GroupKey{group='test-group', key=0}, value=null, metadata=InternalMetadataImpl{actual=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=5, version=40}}, created=-1, lastUsed=-1}, marshaller=org.infinispan.marshall.core.GlobalMarshaller@26ecc0d1} in dummy map store@5d185974
> ---
> 16:02:40,638 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistence[NON_OWNER, SCATTERED_SYNC]
> java.lang.AssertionError:
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.9.9.jar:?]
> at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24) ~[testng-6.9.9.jar:?]
> at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:33) ~[testng-6.9.9.jar:?]
> at org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistence(GetGroupKeysTest.java:162) ~[test-classes/:?]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 10 months
[JBoss JIRA] (ISPN-9224) Reduce ExceptionEvictionTest combinations
by Vittorio Rigamonti (JIRA)
[ https://issues.jboss.org/browse/ISPN-9224?page=com.atlassian.jira.plugin.... ]
Vittorio Rigamonti updated ISPN-9224:
-------------------------------------
Fix Version/s: 9.3.0.Final
(was: 9.3.0.CR1)
> Reduce ExceptionEvictionTest combinations
> -----------------------------------------
>
> Key: ISPN-9224
> URL: https://issues.jboss.org/browse/ISPN-9224
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Core
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.3.0.Final
>
>
> The ExceptionEvictionTest tests all possible permutations. This is a bit wasteful as the multiple node ones should behave the same irrespective of the storage type. The local mode should be sufficient to test different storage type calculations.
> We can comment out quite a few of the tests, which should allow for quicker test runs
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 10 months
[JBoss JIRA] (ISPN-9188) Cancelling the last segment doesn't remove transfer task
by Vittorio Rigamonti (JIRA)
[ https://issues.jboss.org/browse/ISPN-9188?page=com.atlassian.jira.plugin.... ]
Vittorio Rigamonti updated ISPN-9188:
-------------------------------------
Fix Version/s: 9.3.0.Final
(was: 9.3.0.CR1)
> Cancelling the last segment doesn't remove transfer task
> --------------------------------------------------------
>
> Key: ISPN-9188
> URL: https://issues.jboss.org/browse/ISPN-9188
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.3.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 9.3.0.Final
>
>
> If an {{InboundTransferTask}} has some completed segments and the last segment in an is cancelled with {{cancelSegments()}} (because the local node is no longer an owner), the task is never completed or removed from the list of active transfers.
> In the log below, NodeA adds segment 243 in topology 10. The transfer fails because NodeB shut down, so it's retried in topology 11 from NodeC.,
> {noformat}
> 13:19:17,057 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = true, isMember = true, topology = CacheTopology{id=10, phase=READ_OLD_WRITE_ALL, rebalanceId=4, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 79+82, Test-NodeB-56589: 90+84, Test-NodeC-22289: 87+90]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (4)[Test-NodeA-53146: 55+54, Test-NodeB-56589: 67+60, Test-NodeC-22289: 67+69, Test-NodeD-8561: 67+73]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (4)[Test-NodeA-53146: 79+87, Test-NodeB-56589: 90+85, Test-NodeC-22289: 87+93, Test-NodeD-8561: 0+140]}, actualMembers=[Test-NodeA-53146, Test-NodeB-56589, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, 01379420-7999-4abf-abd8-2777454c035a, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,097 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] On cache cluster-listener we have: added segments: {18 71 91-92 243}; removed segments: {}
> 13:19:17,097 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeB-56589 for segments {18 71 91 243}
> 13:19:17,098 TRACE (transport-thread-Test-NodeA-p22712-t2:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {92}
> 13:19:17,107 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = false, isMember = true, topology = CacheTopology{id=11, phase=READ_OLD_WRITE_ALL, rebalanceId=4, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+39, Test-NodeC-22289: 134+43]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 120+31, Test-NodeC-22289: 136+42]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+60, Test-NodeC-22289: 134+71]}, actualMembers=[Test-NodeA-53146, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,140 TRACE (stateTransferExecutor-thread-Test-NodeA-p22713-t1:[StateRequest-cluster-listener]) [StateConsumerImpl] Inbound transfer finished: InboundTransferTask{segments={18 71 91 243}, finishedSegments={}, unfinishedSegments={18 71 91 243}, source=Test-NodeB-56589, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@5237f653[Completed exceptionally], topologyId=10, timeout=240000, cacheName=cluster-listener}
> 13:19:17,156 TRACE (remote-thread-Test-NodeA-p22710-t5:[cluster-listener]) [StateConsumerImpl] Segments not received yet for cache cluster-listener: {Test-NodeB-56589=[InboundTransferTask{segments={18 71 91 243}, finishedSegments={}, unfinishedSegments={18 71 91 243}, source=Test-NodeB-56589, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@5237f653[Completed exceptionally], topologyId=10, timeout=240000, cacheName=cluster-listener}]}
> 13:19:17,273 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Removing inbound transfers from node Test-NodeB-56589 for segments {18 71 91 243}
> 13:19:17,274 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {18 71 91 243}
> 13:19:17,283 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Received new topology for cache cluster-listener, isRebalance = true, isMember = true, topology = CacheTopology{id=12, phase=READ_OLD_WRITE_ALL, rebalanceId=5, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (2)[Test-NodeA-53146: 122+39, Test-NodeC-22289: 134+43]}, pendingCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 76+78, Test-NodeC-22289: 90+80, Test-NodeD-8561: 90+98]}, unionCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (3)[Test-NodeA-53146: 122+75, Test-NodeC-22289: 134+73, Test-NodeD-8561: 0+188]}, actualMembers=[Test-NodeA-53146, Test-NodeC-22289, Test-NodeD-8561], persistentUUIDs=[dff2268e-e59e-48cc-809d-dcfe0774d1d8, e9090b94-dc69-49d8-a2dc-ae0ad7f18fe0, 8edade14-6827-4b67-93d2-49b93844d8ad]}
> 13:19:17,285 WARN (remote-thread-Test-NodeA-p22710-t6:[cluster-listener]) [StateConsumerImpl] Discarding received cache entries for segment 243 of cache cluster-listener because they do not belong to this node.
> 13:19:17,286 TRACE (remote-thread-Test-NodeA-p22710-t6:[cluster-listener]) [StateConsumerImpl] Segments not received yet for cache cluster-listener: {Test-NodeC-22289=[InboundTransferTask{segments={18 71 91 243}, finishedSegments={18 71 91}, unfinishedSegments={243}, source=Test-NodeC-22289, isCancelled=false, completionFuture=java.util.concurrent.CompletableFuture@2755cd[Not completed, 1 dependents], topologyId=11, timeout=240000, cacheName=cluster-listener}]}
> 13:19:17,428 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] On cache cluster-listener we have: added segments: {45 54-55 63 90 95-98 114-115 119 138-140 155-156 167 174-175 205 220-222}; removed segments: {30 66-67 77 93 190-192 243}
> 13:19:17,429 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [InboundTransferTask] Partially cancelling inbound state transfer from node Test-NodeC-22289, segments {243}
> 13:19:17,429 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] Adding transfer from Test-NodeC-22289 for segments {45 54-55 63 90 95-98 114-115 119 138-140 155-156 167 174-175 205 220-222}
> 13:19:17,430 TRACE (transport-thread-Test-NodeA-p22712-t1:[Topology-cluster-listener]) [StateConsumerImpl] notifyEndOfStateTransferIfNeeded: no active transfers
> {noformat}
> The last message is incorrect: the rebalance phase is not confirmed because there are 2 active transfers from NodeC: the 243 one, which wasn't completely cancelled, and the 45...222 one, which hasn't yet started.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 10 months