[
https://issues.redhat.com/browse/ISPN-12423?page=com.atlassian.jira.plugi...
]
Evgenii Balakhonov commented on ISPN-12423:
-------------------------------------------
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype
or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached
cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default
(self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8
waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a
java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at
java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at
org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at
org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at
org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads
more than critical bound.
As a temporary patch I changed class
org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
try {
return Objects.requireNonNull(future, "Completable Future must be
non-null.")
.get(10, TimeUnit.SECONDS);
} catch (TimeoutException tme) {
future.cancel(true);
return null;
} catch (ExecutionException e) {
Throwable cause = e.getCause();
cause.addSuppressed(new TraceException());
throw cause;
}
}
It works fine, but it is very dirty solution ;)
Infispan thread freezing (STUCK) - dead lock occured
----------------------------------------------------
Key: ISPN-12423
URL:
https://issues.redhat.com/browse/ISPN-12423
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 11.0.4.Final
Reporter: Evgenii Balakhonov
Priority: Major
Attachments: Thread dump - heap storage.txt, Thread dump sync.txt, Thread
dump.txt, cache-configuration-jdbc.xml, cache-configuration-jdbc.xml
During huge load some threads hangs on
java.lang.Thread.State: WAITING (parking)
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005d2e654a8> (a
java.util.concurrent.locks.StampedLock)
at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215)
at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428)
at
org.infinispan.container.offheap.OffHeapConcurrentMap.peekOrGet(OffHeapConcurrentMap.java:615)
at
org.infinispan.container.offheap.OffHeapConcurrentMap.peek(OffHeapConcurrentMap.java:682)
I attached Infinispan configuration and three thread dumps:
* off heap storage enabled (Thread dump.txt)
* heap storage enabled (Thread dump - heap storage.txt)
* off heap storage enabled and replicated cache mode="SYNC" (thread dump
sync.txt)
Under high load, Infinspan freezes 100% of the cases.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)