[JBoss JIRA] (ISPN-6308) Stream iterator method can fail sometimes with a node failure
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-6308?page=com.atlassian.jira.plugin.... ]
Work on ISPN-6308 started by William Burns.
-------------------------------------------
> Stream iterator method can fail sometimes with a node failure
> -------------------------------------------------------------
>
> Key: ISPN-6308
> URL: https://issues.jboss.org/browse/ISPN-6308
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 8.1.2.Final
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 8.2.0.Final
>
>
> While adding some additional changes for replication using distributed streams I ran the stress test. I found that iterator could fail somehow. Need to patch this issue.
> Using following settings:
> {code}
> protected final static int CACHE_COUNT = 5;
> protected final static int THREAD_MULTIPLIER = 15;
> protected final static long CACHE_ENTRY_COUNT = 5000;
> {code}
> {code}
> 15:25:29,655 FATAL (ForkThread-17,DistributedStreamRehashStressTest) [DistributedStreamRehashStressTest] Exception encountered:
> java.lang.IllegalArgumentException: Thread[ForkThread-17,DistributedStreamRehashStressTest,5,main]-Found incorrect amount 6 of entries, expected 10 for segment 39 missing entries [ImmortalCacheEntry{key=1922, value=1922}, ImmortalCacheEntry{key=2107, value=2107}, ImmortalCacheEntry{key=2265, value=2265}, ImmortalCacheEntry{key=2412, value=2412}, ImmortalCacheEntry{key=2446, value=2446}, ImmortalCacheEntry{key=2575, value=2575}, ImmortalCacheEntry{key=3315, value=3315}, ImmortalCacheEntry{key=3324, value=3324}, ImmortalCacheEntry{key=3349, value=3349}, ImmortalCacheEntry{key=3675, value=3675}] on iteration 3
> at org.infinispan.stream.stress.DistributedStreamRehashStressTest.lambda$testStressNodesLeavingWhileMultipleIterators$4(DistributedStreamRehashStressTest.java:139)
> at org.infinispan.stream.stress.DistributedStreamRehashStressTest.lambda$testStressNodesLeavingWhilePerformingCallable$5(DistributedStreamRehashStressTest.java:173)
> at org.infinispan.test.AbstractInfinispanTest$LoggingCallable.call(AbstractInfinispanTest.java:478)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-6308) Stream iterator method can fail sometimes with a node failure
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-6308?page=com.atlassian.jira.plugin.... ]
William Burns updated ISPN-6308:
--------------------------------
Fix Version/s: 8.2.0.Final
> Stream iterator method can fail sometimes with a node failure
> -------------------------------------------------------------
>
> Key: ISPN-6308
> URL: https://issues.jboss.org/browse/ISPN-6308
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 8.1.2.Final
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 8.2.0.Final
>
>
> While adding some additional changes for replication using distributed streams I ran the stress test. I found that iterator could fail somehow. Need to patch this issue.
> Using following settings:
> {code}
> protected final static int CACHE_COUNT = 5;
> protected final static int THREAD_MULTIPLIER = 15;
> protected final static long CACHE_ENTRY_COUNT = 5000;
> {code}
> {code}
> 15:25:29,655 FATAL (ForkThread-17,DistributedStreamRehashStressTest) [DistributedStreamRehashStressTest] Exception encountered:
> java.lang.IllegalArgumentException: Thread[ForkThread-17,DistributedStreamRehashStressTest,5,main]-Found incorrect amount 6 of entries, expected 10 for segment 39 missing entries [ImmortalCacheEntry{key=1922, value=1922}, ImmortalCacheEntry{key=2107, value=2107}, ImmortalCacheEntry{key=2265, value=2265}, ImmortalCacheEntry{key=2412, value=2412}, ImmortalCacheEntry{key=2446, value=2446}, ImmortalCacheEntry{key=2575, value=2575}, ImmortalCacheEntry{key=3315, value=3315}, ImmortalCacheEntry{key=3324, value=3324}, ImmortalCacheEntry{key=3349, value=3349}, ImmortalCacheEntry{key=3675, value=3675}] on iteration 3
> at org.infinispan.stream.stress.DistributedStreamRehashStressTest.lambda$testStressNodesLeavingWhileMultipleIterators$4(DistributedStreamRehashStressTest.java:139)
> at org.infinispan.stream.stress.DistributedStreamRehashStressTest.lambda$testStressNodesLeavingWhilePerformingCallable$5(DistributedStreamRehashStressTest.java:173)
> at org.infinispan.test.AbstractInfinispanTest$LoggingCallable.call(AbstractInfinispanTest.java:478)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-6308) Stream iterator method can fail sometimes with a node failure
by William Burns (JIRA)
William Burns created ISPN-6308:
-----------------------------------
Summary: Stream iterator method can fail sometimes with a node failure
Key: ISPN-6308
URL: https://issues.jboss.org/browse/ISPN-6308
Project: Infinispan
Issue Type: Bug
Components: Distributed Execution and Map/Reduce
Affects Versions: 8.1.2.Final
Reporter: William Burns
Assignee: William Burns
While adding some additional changes for replication using distributed streams I ran the stress test. I found that iterator could fail somehow. Need to patch this issue.
Using following settings:
{code}
protected final static int CACHE_COUNT = 5;
protected final static int THREAD_MULTIPLIER = 15;
protected final static long CACHE_ENTRY_COUNT = 5000;
{code}
{code}
15:25:29,655 FATAL (ForkThread-17,DistributedStreamRehashStressTest) [DistributedStreamRehashStressTest] Exception encountered:
java.lang.IllegalArgumentException: Thread[ForkThread-17,DistributedStreamRehashStressTest,5,main]-Found incorrect amount 6 of entries, expected 10 for segment 39 missing entries [ImmortalCacheEntry{key=1922, value=1922}, ImmortalCacheEntry{key=2107, value=2107}, ImmortalCacheEntry{key=2265, value=2265}, ImmortalCacheEntry{key=2412, value=2412}, ImmortalCacheEntry{key=2446, value=2446}, ImmortalCacheEntry{key=2575, value=2575}, ImmortalCacheEntry{key=3315, value=3315}, ImmortalCacheEntry{key=3324, value=3324}, ImmortalCacheEntry{key=3349, value=3349}, ImmortalCacheEntry{key=3675, value=3675}] on iteration 3
at org.infinispan.stream.stress.DistributedStreamRehashStressTest.lambda$testStressNodesLeavingWhileMultipleIterators$4(DistributedStreamRehashStressTest.java:139)
at org.infinispan.stream.stress.DistributedStreamRehashStressTest.lambda$testStressNodesLeavingWhilePerformingCallable$5(DistributedStreamRehashStressTest.java:173)
at org.infinispan.test.AbstractInfinispanTest$LoggingCallable.call(AbstractInfinispanTest.java:478)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5042) Remote gets caused by writes could be replicated only to the primary owner
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-5042?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec commented on ISPN-5042:
-------------------------------------------
Hey [~dan.berindei]! Is this issue still valid? I'm interested in this topic so I would be happy to implement some features here...
> Remote gets caused by writes could be replicated only to the primary owner
> --------------------------------------------------------------------------
>
> Key: ISPN-5042
> URL: https://issues.jboss.org/browse/ISPN-5042
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, State Transfer
> Affects Versions: 7.1.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Sebastian Łaskawiec
> Priority: Minor
> Labels: 7.0
> Fix For: 8.2.0.Final
>
>
> For write operations that need the previous value, a write CH-only owner that doesn't have a key locally will attempt to retrieve the key from the read CH-owners.
> Sending the remote get command to all the previous owners will create extra load on the cluster during state transfer, so it should be more efficient to send the remote get only to the primary owner. Even though the latency of some write operations will be higher, the average latency should be better.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5042) Remote gets caused by writes could be replicated only to the primary owner
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-5042?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec reassigned ISPN-5042:
-----------------------------------------
Assignee: Sebastian Łaskawiec
> Remote gets caused by writes could be replicated only to the primary owner
> --------------------------------------------------------------------------
>
> Key: ISPN-5042
> URL: https://issues.jboss.org/browse/ISPN-5042
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, State Transfer
> Affects Versions: 7.1.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Sebastian Łaskawiec
> Priority: Minor
> Labels: 7.0
> Fix For: 8.2.0.Final
>
>
> For write operations that need the previous value, a write CH-only owner that doesn't have a key locally will attempt to retrieve the key from the read CH-owners.
> Sending the remote get command to all the previous owners will create extra load on the cluster during state transfer, so it should be more efficient to send the remote get only to the primary owner. Even though the latency of some write operations will be higher, the average latency should be better.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5238) Maintain session in ispn-mgmt-console (ctrl+R)
by Vladimir Blagojevic (JIRA)
[ https://issues.jboss.org/browse/ISPN-5238?page=com.atlassian.jira.plugin.... ]
Vladimir Blagojevic resolved ISPN-5238.
---------------------------------------
Release Notes Text: Integrated with https://github.com/infinispan/infinispan-management-console/commit/cb99bc...
Resolution: Done
> Maintain session in ispn-mgmt-console (ctrl+R)
> ----------------------------------------------
>
> Key: ISPN-5238
> URL: https://issues.jboss.org/browse/ISPN-5238
> Project: Infinispan
> Issue Type: Task
> Components: JMX, reporting and management
> Reporter: Tomas Sykora
> Assignee: Vladimir Blagojevic
> Fix For: 8.2.0.Final
>
>
> When you reload a page in Infinispan Management Console data is lost and you need to re-login. We need to make sure that the session is maintained properly and operations like ctrl+R, F5 etc. does not affect behaviour negatively.
> It will also help with development and testing when you need to reload a page to see some changes from time to time.
> I did a small research around this topic and made some experiments but I was not able to achieve what I wanted. Maybe we can come up with even different approach.
> Anyway I suppose we can stick to the idea that ctrl+R should just reload the page and data successfully.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5495) ConcurrentStartTest.testConcurrentStart random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-5495?page=com.atlassian.jira.plugin.... ]
Dan Berindei reopened ISPN-5495:
--------------------------------
After the fix, the test sometimes hangs, logging this exception:
{noformat}
07:22:18,670 ERROR (transport-thread-NodeA-p34708-t1) [ClusterTopologyManagerImpl] ISPN000196: Failed to recover cluster state after the current node became the coordinator
java.lang.NullPointerException
at org.infinispan.topology.ClusterTopologyManagerImpl.addResponseValue(ClusterTopologyManagerImpl.java:647) ~[classes/:?]
at org.infinispan.topology.ClusterTopologyManagerImpl.extractResponseValues(ClusterTopologyManagerImpl.java:637) ~[classes/:?]
at org.infinispan.topology.ClusterTopologyManagerImpl.executeOnClusterSync(ClusterTopologyManagerImpl.java:502) ~[classes/:?]
at org.infinispan.topology.ClusterTopologyManagerImpl.recoverClusterStatus(ClusterTopologyManagerImpl.java:388) ~[classes/:?]
at org.infinispan.topology.ClusterTopologyManagerImpl.handleClusterView(ClusterTopologyManagerImpl.java:322) [classes/:?]
at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$start$206(ClusterTopologyManagerImpl.java:132) [classes/:?]
at org.infinispan.topology.ClusterTopologyManagerImpl$$Lambda$15/1651866807.call(Unknown Source) [classes/:?]
{noformat}
> ConcurrentStartTest.testConcurrentStart random failures
> -------------------------------------------------------
>
> Key: ISPN-5495
> URL: https://issues.jboss.org/browse/ISPN-5495
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.2.0.CR1
>
>
> {noformat}
> org.testng.internal.thread.ThreadTimeoutException: Method org.testng.internal.TestNGMethod.testConcurrentStart() didn't finish within the time-out 60000
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
> at org.infinispan.test.TestingUtil.waitForRehashToComplete(TestingUtil.java:253)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month