[Red Hat JIRA] (ISPN-12343) Transport stack default value always set to `tcp`
by Durgesh Anaokar (Jira)
[ https://issues.redhat.com/browse/ISPN-12343?page=com.atlassian.jira.plugi... ]
Durgesh Anaokar updated ISPN-12343:
-----------------------------------
Git Pull Request: https://github.com/infinispan/infinispan/pull/8782
> Transport stack default value always set to `tcp`
> -------------------------------------------------
>
> Key: ISPN-12343
> URL: https://issues.redhat.com/browse/ISPN-12343
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 12.0.0.Dev03, 11.0.3.Final
> Reporter: Ryan Emerson
> Assignee: Tristan Tarrant
> Priority: Major
>
> The default transport configuration in the server is the following:
> {code:java}
> <cache-container name="default" statistics="true">
> <transport cluster="${infinispan.cluster.name}" stack="${infinispan.cluster.stack:tcp}" node-name="${infinispan.node.name:}"/>
> </cache-container>
> {code}
> If the xml is updated so that {{ stack="${infinispan.cluster.stack:udp}" }} and the property is not set on launch, the "tcp" stack is still loaded instead of "udp".
> Conversely, simply specifying {{stack="udp"}} works as expected, with the "udp" stack loaded.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12428) Upgrade JUnit to 4.13.1
by Pedro Ruivo (Jira)
Pedro Ruivo created ISPN-12428:
----------------------------------
Summary: Upgrade JUnit to 4.13.1
Key: ISPN-12428
URL: https://issues.redhat.com/browse/ISPN-12428
Project: Infinispan
Issue Type: Component Upgrade
Affects Versions: 12.0.0.Dev04, 11.0.4.Final
Reporter: Pedro Ruivo
Assignee: Pedro Ruivo
Fix For: 12.0.0.Dev05, 11.0.5.Final
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12423) Infispan thread freezing (STUCK) - dead lock occured
by Evgenii Balakhonov (Jira)
[ https://issues.redhat.com/browse/ISPN-12423?page=com.atlassian.jira.plugi... ]
Evgenii Balakhonov edited comment on ISPN-12423 at 10/14/20 7:58 AM:
---------------------------------------------------------------------
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
was (Author: c0ff75):
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed method get() of class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
try {
return Objects.requireNonNull(future, "Completable Future must be non-null.")
.get(10, TimeUnit.SECONDS);
} catch (TimeoutException tme) {
future.cancel(true);
return null;
} catch (ExecutionException e) {
Throwable cause = e.getCause();
cause.addSuppressed(new TraceException());
throw cause;
}
}
It works fine, but it is very dirty solution ;)
> Infispan thread freezing (STUCK) - dead lock occured
> ----------------------------------------------------
>
> Key: ISPN-12423
> URL: https://issues.redhat.com/browse/ISPN-12423
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 11.0.4.Final
> Reporter: Evgenii Balakhonov
> Priority: Major
> Attachments: Thread dump - heap storage.txt, Thread dump sync.txt, Thread dump.txt, cache-configuration-jdbc.xml, cache-configuration-jdbc.xml
>
>
> During huge load some threads hangs on
> java.lang.Thread.State: WAITING (parking)
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000005d2e654a8> (a java.util.concurrent.locks.StampedLock)
> at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215)
> at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peekOrGet(OffHeapConcurrentMap.java:615)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peek(OffHeapConcurrentMap.java:682)
> I attached Infinispan configuration and three thread dumps:
> * off heap storage enabled (Thread dump.txt)
> * heap storage enabled (Thread dump - heap storage.txt)
> * off heap storage enabled and replicated cache mode="SYNC" (thread dump sync.txt)
> Under high load, Infinspan freezes 100% of the cases.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12423) Infispan thread freezing (STUCK) - dead lock occured
by Evgenii Balakhonov (Jira)
[ https://issues.redhat.com/browse/ISPN-12423?page=com.atlassian.jira.plugi... ]
Evgenii Balakhonov edited comment on ISPN-12423 at 10/14/20 6:57 AM:
---------------------------------------------------------------------
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed method get() of class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
try {
return Objects.requireNonNull(future, "Completable Future must be non-null.")
.get(10, TimeUnit.SECONDS);
} catch (TimeoutException tme) {
future.cancel(true);
return null;
} catch (ExecutionException e) {
Throwable cause = e.getCause();
cause.addSuppressed(new TraceException());
throw cause;
}
}
It works fine, but it is very dirty solution ;)
was (Author: c0ff75):
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
_*try*_
_*{*_
_*return Objects.requireNonNull(future, "Completable Future must be non-null.") .get(10, TimeUnit.SECONDS); }*_
_*catch (TimeoutException tme)*_
_*{*_
_*future.cancel(true); return null;*_
_*}*_
catch (ExecutionException e)
{ Throwable cause = e.getCause(); cause.addSuppressed(new TraceException()); throw cause; }
}
It works fine, but it is very dirty solution ;)
> Infispan thread freezing (STUCK) - dead lock occured
> ----------------------------------------------------
>
> Key: ISPN-12423
> URL: https://issues.redhat.com/browse/ISPN-12423
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 11.0.4.Final
> Reporter: Evgenii Balakhonov
> Priority: Major
> Attachments: Thread dump - heap storage.txt, Thread dump sync.txt, Thread dump.txt, cache-configuration-jdbc.xml, cache-configuration-jdbc.xml
>
>
> During huge load some threads hangs on
> java.lang.Thread.State: WAITING (parking)
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000005d2e654a8> (a java.util.concurrent.locks.StampedLock)
> at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215)
> at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peekOrGet(OffHeapConcurrentMap.java:615)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peek(OffHeapConcurrentMap.java:682)
> I attached Infinispan configuration and three thread dumps:
> * off heap storage enabled (Thread dump.txt)
> * heap storage enabled (Thread dump - heap storage.txt)
> * off heap storage enabled and replicated cache mode="SYNC" (thread dump sync.txt)
> Under high load, Infinspan freezes 100% of the cases.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12423) Infispan thread freezing (STUCK) - dead lock occured
by Evgenii Balakhonov (Jira)
[ https://issues.redhat.com/browse/ISPN-12423?page=com.atlassian.jira.plugi... ]
Evgenii Balakhonov edited comment on ISPN-12423 at 10/14/20 6:56 AM:
---------------------------------------------------------------------
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
_*try*_
_*{*_
_*return Objects.requireNonNull(future, "Completable Future must be non-null.") .get(10, TimeUnit.SECONDS); }*_
_*catch (TimeoutException tme)*_
_*{*_
_*future.cancel(true); return null;*_
_*}*_
catch (ExecutionException e)
{ Throwable cause = e.getCause(); cause.addSuppressed(new TraceException()); throw cause; }
}
It works fine, but it is very dirty solution ;)
was (Author: c0ff75):
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
_*try {
return Objects.requireNonNull(future, "Completable Future must be non-null.")
.get(10, TimeUnit.SECONDS);
} catch (TimeoutException tme) {
future.cancel(true);
return null;
} *_catch (ExecutionException e) {
Throwable cause = e.getCause();
cause.addSuppressed(new TraceException());
throw cause;
}
}
It works fine, but it is very dirty solution ;)
> Infispan thread freezing (STUCK) - dead lock occured
> ----------------------------------------------------
>
> Key: ISPN-12423
> URL: https://issues.redhat.com/browse/ISPN-12423
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 11.0.4.Final
> Reporter: Evgenii Balakhonov
> Priority: Major
> Attachments: Thread dump - heap storage.txt, Thread dump sync.txt, Thread dump.txt, cache-configuration-jdbc.xml, cache-configuration-jdbc.xml
>
>
> During huge load some threads hangs on
> java.lang.Thread.State: WAITING (parking)
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000005d2e654a8> (a java.util.concurrent.locks.StampedLock)
> at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215)
> at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peekOrGet(OffHeapConcurrentMap.java:615)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peek(OffHeapConcurrentMap.java:682)
> I attached Infinispan configuration and three thread dumps:
> * off heap storage enabled (Thread dump.txt)
> * heap storage enabled (Thread dump - heap storage.txt)
> * off heap storage enabled and replicated cache mode="SYNC" (thread dump sync.txt)
> Under high load, Infinspan freezes 100% of the cases.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12423) Infispan thread freezing (STUCK) - dead lock occured
by Evgenii Balakhonov (Jira)
[ https://issues.redhat.com/browse/ISPN-12423?page=com.atlassian.jira.plugi... ]
Evgenii Balakhonov edited comment on ISPN-12423 at 10/14/20 6:55 AM:
---------------------------------------------------------------------
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
_*try {
return Objects.requireNonNull(future, "Completable Future must be non-null.")
.get(10, TimeUnit.SECONDS);
} catch (TimeoutException tme) {
future.cancel(true);
return null;
} *_catch (ExecutionException e) {
Throwable cause = e.getCause();
cause.addSuppressed(new TraceException());
throw cause;
}
}
It works fine, but it is very dirty solution ;)
was (Author: c0ff75):
Hello, Will Burns
I can reproduce this problem in any time. I can show it if you want, for example via Skype or TeamViewer
And moreover. If I switch off OFF_HEAP and SYNC (see last attached cache-configuration-jdbc.xml) I have other problem with dead lock:
"[STUCK] ExecuteThread: '220' for queue: 'weblogic.kernel.Default (self-tuning)'" #812 daemon prio=1 os_prio=0 tid=0x00007ff29c136000 nid=0x4fb8 waiting on condition [0x00007ff1e281f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006ec18b8a8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
See attached thread dump "Thread dump - heap storage.txt".
As a result my WebLogic server going to automatic restart because count of STUCK threads more than critical bound.
As a temporary patch I changed class org.infinispan.interceptors.impl.SimpleAsyncInvocationStage:
@Override
public Object get() throws Throwable {
try {
return Objects.requireNonNull(future, "Completable Future must be non-null.")
.get(10, TimeUnit.SECONDS);
} catch (TimeoutException tme) {
future.cancel(true);
return null;
} catch (ExecutionException e) {
Throwable cause = e.getCause();
cause.addSuppressed(new TraceException());
throw cause;
}
}
It works fine, but it is very dirty solution ;)
> Infispan thread freezing (STUCK) - dead lock occured
> ----------------------------------------------------
>
> Key: ISPN-12423
> URL: https://issues.redhat.com/browse/ISPN-12423
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 11.0.4.Final
> Reporter: Evgenii Balakhonov
> Priority: Major
> Attachments: Thread dump - heap storage.txt, Thread dump sync.txt, Thread dump.txt, cache-configuration-jdbc.xml, cache-configuration-jdbc.xml
>
>
> During huge load some threads hangs on
> java.lang.Thread.State: WAITING (parking)
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000005d2e654a8> (a java.util.concurrent.locks.StampedLock)
> at java.util.concurrent.locks.StampedLock.acquireRead(StampedLock.java:1215)
> at java.util.concurrent.locks.StampedLock.readLock(StampedLock.java:428)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peekOrGet(OffHeapConcurrentMap.java:615)
> at org.infinispan.container.offheap.OffHeapConcurrentMap.peek(OffHeapConcurrentMap.java:682)
> I attached Infinispan configuration and three thread dumps:
> * off heap storage enabled (Thread dump.txt)
> * heap storage enabled (Thread dump - heap storage.txt)
> * off heap storage enabled and replicated cache mode="SYNC" (thread dump sync.txt)
> Under high load, Infinspan freezes 100% of the cases.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months