[JBoss JIRA] (ISPN-8525) StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync failing randomly
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-8525?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-8525:
-----------------------------------
Affects Version/s: 9.2.0.Alpha2
> StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync failing randomly
> -------------------------------------------------------------------------
>
> Key: ISPN-8525
> URL: https://issues.jboss.org/browse/ISPN-8525
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.0.Alpha2
> Reporter: Galder Zamarreño
> Labels: testsuite_stability
>
> http://ci.infinispan.org/job/Infinispan/job/PR-5556/17/
> org.infinispan.statetransfer.StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync
> {code}
> Error Message
> Timed out waiting for rebalancing to complete on node StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, expected member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeB-9821], current member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365]!
> Stacktrace
> java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, expected member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeB-9821], current member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365]!
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:385)
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:421)
> at org.infinispan.statetransfer.StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync(StaleLocksWithLockOnlyTxDuringStateTransferTest.java:102)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ... Removed 16 stack frames
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8524) ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5 failing randomly
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-8524?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-8524:
-----------------------------------
Affects Version/s: 9.2.0.Alpha2
> ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5 failing randomly
> --------------------------------------------------------------------------------------
>
> Key: ISPN-8524
> URL: https://issues.jboss.org/browse/ISPN-8524
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.0.Alpha2
> Reporter: Galder Zamarreño
> Assignee: Radim Vansa
> Labels: testsuite_stability
>
> http://ci.infinispan.org/job/Infinispan/job/PR-5556/17/
> org.infinispan.partitionhandling.ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5[SCATTERED_SYNC]
> {code}
> java.util.concurrent.TimeoutException
> at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
> at org.infinispan.partitionhandling.ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate(ScatteredDelayedAvailabilityUpdateTest.java:75)
> at org.infinispan.partitionhandling.DelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5(DelayedAvailabilityUpdateTest.java:39)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ... Removed 16 stack frames
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8525) StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync failing randomly
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-8525:
--------------------------------------
Summary: StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync failing randomly
Key: ISPN-8525
URL: https://issues.jboss.org/browse/ISPN-8525
Project: Infinispan
Issue Type: Bug
Components: Core
Reporter: Galder Zamarreño
http://ci.infinispan.org/job/Infinispan/job/PR-5556/17/
org.infinispan.statetransfer.StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync
{code}
Error Message
Timed out waiting for rebalancing to complete on node StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, expected member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeB-9821], current member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365]!
Stacktrace
java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, expected member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeB-9821], current member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365]!
at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:385)
at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:421)
at org.infinispan.statetransfer.StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync(StaleLocksWithLockOnlyTxDuringStateTransferTest.java:102)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
... Removed 16 stack frames
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8524) ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5 failing randomly
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-8524:
--------------------------------------
Summary: ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5 failing randomly
Key: ISPN-8524
URL: https://issues.jboss.org/browse/ISPN-8524
Project: Infinispan
Issue Type: Bug
Components: Core
Reporter: Galder Zamarreño
Assignee: Radim Vansa
http://ci.infinispan.org/job/Infinispan/job/PR-5556/17/
org.infinispan.partitionhandling.ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5[SCATTERED_SYNC]
{code}
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.infinispan.partitionhandling.ScatteredDelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate(ScatteredDelayedAvailabilityUpdateTest.java:75)
at org.infinispan.partitionhandling.DelayedAvailabilityUpdateTest.testDelayedAvailabilityUpdate5(DelayedAvailabilityUpdateTest.java:39)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
... Removed 16 stack frames
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-8411:
-----------------------------------
Affects Version/s: 9.2.0.Alpha2
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final, 9.2.0.Alpha2
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Radim Vansa
> Fix For: 9.2.0.Beta1, 9.2.0.Final
>
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-8411:
-----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.2.0.Beta1
9.2.0.Final
Resolution: Done
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final, 9.2.0.Alpha2
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Radim Vansa
> Fix For: 9.2.0.Beta1, 9.2.0.Final
>
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8523) DistSyncOnePhaseWriteSkewTxStateTransferTest.testStateTransferWithClusterIdle failing randomly
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-8523:
--------------------------------------
Summary: DistSyncOnePhaseWriteSkewTxStateTransferTest.testStateTransferWithClusterIdle failing randomly
Key: ISPN-8523
URL: https://issues.jboss.org/browse/ISPN-8523
Project: Infinispan
Issue Type: Enhancement
Components: Core
Affects Versions: 9.2.0.Alpha2
Reporter: Galder Zamarreño
http://ci.infinispan.org/job/Infinispan/job/PR-5539/1/testReport/junit/or...
org.infinispan.xsite.statetransfer.DistSyncOnePhaseWriteSkewTxStateTransferTest.testStateTransferWithClusterIdle[null, tx=false]
{code}
java.lang.AssertionError:
at org.infinispan.xsite.statetransfer.BaseStateTransferTest$20.assertInCache(BaseStateTransferTest.java:601)
at org.infinispan.xsite.AbstractXSiteTest.assertInSite(AbstractXSiteTest.java:172)
at org.infinispan.xsite.statetransfer.BaseStateTransferTest.assertNoStateTransferInReceivingSite(BaseStateTransferTest.java:596)
at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testStateTransferWithClusterIdle(BaseStateTransferTest.java:199)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
... Removed 20 stack frames
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months
[JBoss JIRA] (ISPN-8522) PreferAvailabilityStrategy conflict resolution should occur if an expected member is not in max topology
by Ryan Emerson (JIRA)
Ryan Emerson created ISPN-8522:
----------------------------------
Summary: PreferAvailabilityStrategy conflict resolution should occur if an expected member is not in max topology
Key: ISPN-8522
URL: https://issues.jboss.org/browse/ISPN-8522
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 9.2.0.Alpha2
Reporter: Ryan Emerson
Assignee: Ryan Emerson
Fix For: 9.2.0.Beta1
The current method for determining whether a split brain heal is occurring during an on merge event is wrong. Instead we should assume healing whenever one or more of the expected members is not in the membership of the maxTopology received.
Furthermore, we should also utilise the Union CH of all distinct CHs encountered for conflict resolution. This is necessary to ensure that we read the entries associated with all possible write owners before the rebalance occurs.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 2 months