[JBoss JIRA] (ISPN-8525) StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync failing randomly
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8525?page=com.atlassian.jira.plugin.... ]
Dan Berindei reassigned ISPN-8525:
----------------------------------
Assignee: Dan Berindei
> StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync failing randomly
> -------------------------------------------------------------------------
>
> Key: ISPN-8525
> URL: https://issues.jboss.org/browse/ISPN-8525
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.0.Alpha2
> Reporter: Galder Zamarreño
> Assignee: Dan Berindei
> Labels: testsuite_stability
>
> http://ci.infinispan.org/job/Infinispan/job/PR-5556/17/
> org.infinispan.statetransfer.StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync
> {code}
> Error Message
> Timed out waiting for rebalancing to complete on node StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, expected member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeB-9821], current member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365]!
> Stacktrace
> java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, expected member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365, StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeB-9821], current member list is [StaleLocksWithLockOnlyTxDuringStateTransferTest-NodeA-8365]!
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:385)
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:421)
> at org.infinispan.statetransfer.StaleLocksWithLockOnlyTxDuringStateTransferTest.testSync(StaleLocksWithLockOnlyTxDuringStateTransferTest.java:102)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ... Removed 16 stack frames
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8555) CacheManagerTest.testConcurrentCacheManagerStopAndGetCache randomly hangs
by Dan Berindei (JIRA)
Dan Berindei created ISPN-8555:
----------------------------------
Summary: CacheManagerTest.testConcurrentCacheManagerStopAndGetCache randomly hangs
Key: ISPN-8555
URL: https://issues.jboss.org/browse/ISPN-8555
Project: Infinispan
Issue Type: Bug
Components: Test Suite - Core
Affects Versions: 9.2.0.Beta1
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 9.2.0.Beta2
If there is any exception, the finally block tries to stop the cache manager without first unblocking the stop method, and it hangs:
{noformat}
"ForkThread-1,CacheManagerTest" #204160 prio=5 os_prio=0 tid=0x00007fa1900aa800 nid=0x1be5 waiting on condition [0x00007fa0db5b3000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c846b690> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
at org.infinispan.manager.CacheManagerTest$2.stop(CacheManagerTest.java:274)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:91)
at org.infinispan.commons.util.SecurityActions$$Lambda$169/1215571888.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:83)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:88)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:165)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:883)
at org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:684)
at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:583)
- locked <0x00000000c846b6d8> (a org.infinispan.factories.GlobalComponentRegistry)
at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:271)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:206)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1000)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.manager.CacheManagerTest.lambda$testConcurrentCacheManagerStopAndGetCache$4(CacheManagerTest.java:279)
at org.infinispan.manager.CacheManagerTest$$Lambda$3417/950279155.call(Unknown Source)
at org.infinispan.test.AbstractInfinispanTest$LoggingCallable.call(AbstractInfinispanTest.java:543)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x00000000c846b8d0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"testng-CacheManagerTest" #24 prio=5 os_prio=0 tid=0x00007fa260ece000 nid=0x44b6 waiting on condition [0x00007fa1e4626000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c84702c0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:695)
at org.infinispan.test.TestingUtil.killCacheManagers(TestingUtil.java:774)
at org.infinispan.test.TestingUtil.killCacheManagers(TestingUtil.java:765)
at org.infinispan.manager.CacheManagerTest.testConcurrentCacheManagerStopAndGetCache(CacheManagerTest.java:295)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x00000000c4628978> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"ForkThread-2,CacheManagerTest" #204172 prio=5 os_prio=0 tid=0x00007fa1900f6800 nid=0x1bf2 waiting on condition [0x00007fa0da9a8000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c84181c0> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
at org.infinispan.manager.DefaultCacheManager.terminate(DefaultCacheManager.java:681)
at org.infinispan.manager.DefaultCacheManager.stopCaches(DefaultCacheManager.java:727)
at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:704)
at org.infinispan.manager.CacheManagerTest.lambda$testConcurrentCacheManagerStopAndGetCache$5(CacheManagerTest.java:282)
at org.infinispan.manager.CacheManagerTest$$Lambda$3418/1712334616.run(Unknown Source)
at org.infinispan.test.AbstractInfinispanTest$RunnableWrapper.run(AbstractInfinispanTest.java:510)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x00000000c8418270> (a java.util.concurrent.ThreadPoolExecutor$Worker)
- <0x00000000c84702c0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8554) Automatically interrupt hung tests
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8554?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8554:
-------------------------------
Summary: Automatically interrupt hung tests (was: Automatically interrupt hanged tests)
> Automatically interrupt hung tests
> ----------------------------------
>
> Key: ISPN-8554
> URL: https://issues.jboss.org/browse/ISPN-8554
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.0.Beta2
>
>
> Tests should always use timed wait, e.g. {{future.get(10, SECONDS)}} instead of {{future.get()}}. However, it's really easy to use the wrong method, and if a deadlock happens the test suite will never finish.
> Jenkins builds do have a timeout, but the timeout kill procedure is not at all investigation-friendly: only the console output survives, the logs and test results don't.
> We can add a listener ({{IHookable}} in TestNG, because regular listeners are not always invoked on the same thread as the test method) to detect when a test has been running for too long and interrupt the test thread. We can even save a thread dump to make the investigation easier.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8550) Try to estimate malloc overhead and add to memory based eviction
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8550?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-8550:
------------------------------------
[~william.burns] how did you use Valgrind? I wanted to reproduce your results and I tried the massif tool with {{--pages-as-heap=yes}} to capture the real allocation size, but the output was kind of hard to parse.
Eventually I found that {{$(which time) -f "%E %M"}} writes the peak RSS and I've been testing with that. After modifying the test to skip the warmup and take the size and count of allocations as parameters, this is what I got on my machine:
||alloc size||alloc count||time %M kbytes||est. entries kbytes||est. overhead 8 aligned at 16||diff||est. overhead 0 aligned at 32||diff||est.baseline kbytes||kbyte size||
|100000000|1|170336|97656.25|97656.265625|-0.015625|97656.25|0|72679.75|1024|
|1000000|1000|1053108|976562.5|976578.125|3850.125|976562.5|3865.75|
|1|1000000|104200|976.5625|15625|15895.25|31250|270.25|
|10|1000000|104160|9765.625|31250|230.25|31250|230.25|
|100|1000000|182524|97656.25|109375|469.25|125000|-15155.75|
|104|1000000|182180|101562.5|109375|125.25|125000|-15499.75|
|108|1000000|198268|105468.75|125000|588.25|125000|588.25|
|116|1000000|198060|113281.25|125000|380.25|125000|380.25|
|1000|1000000|1057348|976562.5|984375|293.25|1000000|-15331.75|
|1004|1000000|1073024|980468.75|1000000|344.25|1000000|344.25|
|1008|1000000|1073068|984375|1000000|388.25|1000000|388.25|
|1012|1000000|1073128|988281.25|1000000|448.25|1000000|448.25|
|1016|1000000|1072636|992187.5|1000000|-43.75|1000000|-43.75|
|1020|1000000|1088316|996093.75|1015625|11.25|1000000|15636.25|
|1024|1000000|1088564|1000000|1015625|259.25|1000000|15884.25|
So it looks like on my machine malloc adds 8 bytes and then rounds up the allocation to a multiple of 16, except the minimum size is 32 bytes.
This is my modified test:
{code:java}
public static void main(String[] args) throws InterruptedException {
int allocationSize = Integer.parseInt(args[0]);
int allocationCount = Integer.parseInt(args[1]);
System.out.println(allocationSize + " " + allocationCount);
for (int i = 0; i < allocationCount; ++i) {
long address = OffHeapMemory.INSTANCE.allocate(allocationSize);
for (int j = 0; j < allocationSize; j += 1024) {
OffHeapMemory.INSTANCE.putByte(address, j, (byte) 0);
}
}
// Make sure Valgrind/Massif sees a the peak here
long address = OffHeapMemory.INSTANCE.allocate(1);
OffHeapMemory.INSTANCE.free(address);
Thread.sleep(200);
}
{code}
> Try to estimate malloc overhead and add to memory based eviction
> ----------------------------------------------------------------
>
> Key: ISPN-8550
> URL: https://issues.jboss.org/browse/ISPN-8550
> Project: Infinispan
> Issue Type: Sub-task
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.2.0.Beta2, 9.1.4.Final
>
>
> We should try to also estimate malloc overhead. We could do something like Dan mentioned at https://github.com/infinispan/infinispan/pull/5590#pullrequestreview-7805...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8550) Try to estimate malloc overhead and add to memory based eviction
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8550?page=com.atlassian.jira.plugin.... ]
Dan Berindei edited comment on ISPN-8550 at 11/24/17 3:46 AM:
--------------------------------------------------------------
[~william.burns] how did you use Valgrind? I wanted to reproduce your results and I tried the massif tool with {{--pages-as-heap=yes}} to capture the real allocation size, but the output was kind of hard to parse.
Eventually I found that {{$(which time) -f "%E %M"}} writes the peak RSS and I've been testing with that. After modifying the test to skip the warmup and take the size and count of allocations as parameters, this is what I got on my machine:
||alloc size||alloc count||time %M kbytes||est. entries kbytes||est. overhead 8 aligned at 16||diff||est. overhead 0 aligned at 32||diff||est.baseline kbytes||kbyte size||
|100000000|1|170336|97656.25|97656.265625|-0.015625|97656.25|0|72679.75|1024|
|1000000|1000|1053108|976562.5|976578.125|3850.125|976562.5|3865.75|
|1|1000000|104200|976.5625|15625|15895.25|31250|270.25|
|10|1000000|104160|9765.625|31250|230.25|31250|230.25|
|100|1000000|182524|97656.25|109375|469.25|125000|-15155.75|
|104|1000000|182180|101562.5|109375|125.25|125000|-15499.75|
|108|1000000|198268|105468.75|125000|588.25|125000|588.25|
|116|1000000|198060|113281.25|125000|380.25|125000|380.25|
|1000|1000000|1057348|976562.5|984375|293.25|1000000|-15331.75|
|1004|1000000|1073024|980468.75|1000000|344.25|1000000|344.25|
|1008|1000000|1073068|984375|1000000|388.25|1000000|388.25|
|1012|1000000|1073128|988281.25|1000000|448.25|1000000|448.25|
|1016|1000000|1072636|992187.5|1000000|-43.75|1000000|-43.75|
|1020|1000000|1088316|996093.75|1015625|11.25|1000000|15636.25|
|1024|1000000|1088564|1000000|1015625|259.25|1000000|15884.25|
So it looks like on my machine malloc adds 8 bytes and then rounds up the allocation to a multiple of 16, except the minimum size is 32 bytes, and really big allocations are treated differently.
This is my modified test:
{code:java}
public static void main(String[] args) throws InterruptedException {
int allocationSize = Integer.parseInt(args[0]);
int allocationCount = Integer.parseInt(args[1]);
System.out.println(allocationSize + " " + allocationCount);
for (int i = 0; i < allocationCount; ++i) {
long address = OffHeapMemory.INSTANCE.allocate(allocationSize);
for (int j = 0; j < allocationSize; j += 1024) {
OffHeapMemory.INSTANCE.putByte(address, j, (byte) 0);
}
}
// Make sure Valgrind/Massif sees a the peak here
long address = OffHeapMemory.INSTANCE.allocate(1);
OffHeapMemory.INSTANCE.free(address);
Thread.sleep(200);
}
{code}
was (Author: dan.berindei):
[~william.burns] how did you use Valgrind? I wanted to reproduce your results and I tried the massif tool with {{--pages-as-heap=yes}} to capture the real allocation size, but the output was kind of hard to parse.
Eventually I found that {{$(which time) -f "%E %M"}} writes the peak RSS and I've been testing with that. After modifying the test to skip the warmup and take the size and count of allocations as parameters, this is what I got on my machine:
||alloc size||alloc count||time %M kbytes||est. entries kbytes||est. overhead 8 aligned at 16||diff||est. overhead 0 aligned at 32||diff||est.baseline kbytes||kbyte size||
|100000000|1|170336|97656.25|97656.265625|-0.015625|97656.25|0|72679.75|1024|
|1000000|1000|1053108|976562.5|976578.125|3850.125|976562.5|3865.75|
|1|1000000|104200|976.5625|15625|15895.25|31250|270.25|
|10|1000000|104160|9765.625|31250|230.25|31250|230.25|
|100|1000000|182524|97656.25|109375|469.25|125000|-15155.75|
|104|1000000|182180|101562.5|109375|125.25|125000|-15499.75|
|108|1000000|198268|105468.75|125000|588.25|125000|588.25|
|116|1000000|198060|113281.25|125000|380.25|125000|380.25|
|1000|1000000|1057348|976562.5|984375|293.25|1000000|-15331.75|
|1004|1000000|1073024|980468.75|1000000|344.25|1000000|344.25|
|1008|1000000|1073068|984375|1000000|388.25|1000000|388.25|
|1012|1000000|1073128|988281.25|1000000|448.25|1000000|448.25|
|1016|1000000|1072636|992187.5|1000000|-43.75|1000000|-43.75|
|1020|1000000|1088316|996093.75|1015625|11.25|1000000|15636.25|
|1024|1000000|1088564|1000000|1015625|259.25|1000000|15884.25|
So it looks like on my machine malloc adds 8 bytes and then rounds up the allocation to a multiple of 16, except the minimum size is 32 bytes.
This is my modified test:
{code:java}
public static void main(String[] args) throws InterruptedException {
int allocationSize = Integer.parseInt(args[0]);
int allocationCount = Integer.parseInt(args[1]);
System.out.println(allocationSize + " " + allocationCount);
for (int i = 0; i < allocationCount; ++i) {
long address = OffHeapMemory.INSTANCE.allocate(allocationSize);
for (int j = 0; j < allocationSize; j += 1024) {
OffHeapMemory.INSTANCE.putByte(address, j, (byte) 0);
}
}
// Make sure Valgrind/Massif sees a the peak here
long address = OffHeapMemory.INSTANCE.allocate(1);
OffHeapMemory.INSTANCE.free(address);
Thread.sleep(200);
}
{code}
> Try to estimate malloc overhead and add to memory based eviction
> ----------------------------------------------------------------
>
> Key: ISPN-8550
> URL: https://issues.jboss.org/browse/ISPN-8550
> Project: Infinispan
> Issue Type: Sub-task
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.2.0.Beta2, 9.1.4.Final
>
>
> We should try to also estimate malloc overhead. We could do something like Dan mentioned at https://github.com/infinispan/infinispan/pull/5590#pullrequestreview-7805...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8554) Automatically interrupt hanged tests
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8554?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8554:
-------------------------------
Status: Open (was: New)
> Automatically interrupt hanged tests
> ------------------------------------
>
> Key: ISPN-8554
> URL: https://issues.jboss.org/browse/ISPN-8554
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.0.Beta2
>
>
> Tests should always use timed wait, e.g. {{future.get(10, SECONDS)}} instead of {{future.get()}}. However, it's really easy to use the wrong method, and if a deadlock happens the test suite will never finish.
> Jenkins builds do have a timeout, but the timeout kill procedure is not at all investigation-friendly: only the console output survives, the logs and test results don't.
> We can add a listener ({{IHookable}} in TestNG, because regular listeners are not always invoked on the same thread as the test method) to detect when a test has been running for too long and interrupt the test thread. We can even save a thread dump to make the investigation easier.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8554) Automatically interrupt hanged tests
by Dan Berindei (JIRA)
Dan Berindei created ISPN-8554:
----------------------------------
Summary: Automatically interrupt hanged tests
Key: ISPN-8554
URL: https://issues.jboss.org/browse/ISPN-8554
Project: Infinispan
Issue Type: Task
Components: Test Suite - Core
Affects Versions: 9.2.0.Beta1
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 9.2.0.Beta2
Tests should always use timed wait, e.g. {{future.get(10, SECONDS)}} instead of {{future.get()}}. However, it's really easy to use the wrong method, and if a deadlock happens the test suite will never finish.
Jenkins builds do have a timeout, but the timeout kill procedure is not at all investigation-friendly: only the console output survives, the logs and test results don't.
We can add a listener ({{IHookable}} in TestNG, because regular listeners are not always invoked on the same thread as the test method) to detect when a test has been running for too long and interrupt the test thread. We can even save a thread dump to make the investigation easier.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (ISPN-8452) KUBE_PING doesn't work with JDG 7.2 OpenShift image
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-8452?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-8452.
--------------------------------
Resolution: Done
> KUBE_PING doesn't work with JDG 7.2 OpenShift image
> ---------------------------------------------------
>
> Key: ISPN-8452
> URL: https://issues.jboss.org/browse/ISPN-8452
> Project: Infinispan
> Issue Type: Bug
> Components: Cloud Integrations
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Blocker
>
> It seems that KUBE_PING doesn't work with the latest version of JDG 7.2 OpenShift image. The reason for this is that JDG 7.2 is shipped with upstream KUBE_PING which conflicts with the one provided in the openshift layer.
> The easiest solution is to revert layer ordering {{layers.conf}} file:
> {code}
> echo "layers=base,openshift" > /opt/datagrid/modules/layers.conf
> {code}
> This solution has additional advantage - we can fallback to previous KUBE_PING version (the one provided by the CE Team) by just changing this small line.
> However if the upstream KUBE_PING works fine and we are happy with it, we should remove the old version from the server. I think that's a solution we should take for JDG 8 or maybe sooner - 7.3.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months