May 2018 - jboss-jira - Jboss List Archives

[JBoss JIRA] (WFWIP-6) Color output cannot be disabled by jboss-cli.xml

by Ingo Weiss (JIRA)

[ https://issues.jboss.org/browse/WFWIP-6?page=com.atlassian.jira.plugin.sy... ] Ingo Weiss resolved WFWIP-6. ---------------------------- Resolution: Done > Color output cannot be disabled by jboss-cli.xml > ------------------------------------------------ > > Key: WFWIP-6 > URL: https://issues.jboss.org/browse/WFWIP-6 > Project: WildFly WIP > Issue Type: Bug > Reporter: Erich Duda > Assignee: Ingo Weiss > Labels: WFCORE-3577 > > When I disabled color output in {{jboss-cli.xml}} using following configuration, the colors were still displayed. > {code:xml} > <color-output> > <enabled>false</enabled> > </color-output> > {code} -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (WFWIP-7) CLI argument --no-color-output has no precedence before XML configuration

by Ingo Weiss (JIRA)

[ https://issues.jboss.org/browse/WFWIP-7?page=com.atlassian.jira.plugin.sy... ] Ingo Weiss resolved WFWIP-7. ---------------------------- Resolution: Done > CLI argument --no-color-output has no precedence before XML configuration > ------------------------------------------------------------------------- > > Key: WFWIP-7 > URL: https://issues.jboss.org/browse/WFWIP-7 > Project: WildFly WIP > Issue Type: Bug > Reporter: Erich Duda > Assignee: Ingo Weiss > Labels: WFCORE-3577 > > When I uncommented {{color-output}} configuration in jboss-cli.xml where {{enabled}} option is set on {{true}}, the {{--no-color-output}} option had not effect. The colors were displayed. > {code:xml|title=jboss-cli.xml} > <color-output> > <enabled>true</enabled> > <error-color>red</error-color> > <warn-color>yellow</warn-color> > <success-color>default</success-color> > <required-color>magenta</required-color> > <workflow-color>default</workflow-color> > <prompt-color>default</prompt-color> > </color-output> > {code} > {code:title=Bash command} > $ ./jboss-cli.sh --no-color-output > {code} > In this case the {{--no-color-output}} should have precedence and no colors should be displayed. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (DROOLS-2521) Implement FEEL functions in preparation for DMN 1.2

by Edson Tirelli (JIRA)

[ https://issues.jboss.org/browse/DROOLS-2521?page=com.atlassian.jira.plugi... ] Edson Tirelli updated DROOLS-2521: ---------------------------------- Sprint: 2018 Week 17-18 > Implement FEEL functions in preparation for DMN 1.2 > --------------------------------------------------- > > Key: DROOLS-2521 > URL: https://issues.jboss.org/browse/DROOLS-2521 > Project: Drools > Issue Type: Enhancement > Components: dmn engine > Affects Versions: 7.7.0.Final > Reporter: Edson Tirelli > Assignee: Fedor Gavrilov > Fix For: 7.8.0.Final > > Attachments: DMN12-190_proposal_v4.docx > > > There are a few new FEEL functions defined for DMN 1.2. > See attached document for the list of functions. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (DROOLS-2521) Implement FEEL functions in preparation for DMN 1.2

by Edson Tirelli (JIRA)

[ https://issues.jboss.org/browse/DROOLS-2521?page=com.atlassian.jira.plugi... ] Edson Tirelli reassigned DROOLS-2521: ------------------------------------- Assignee: Fedor Gavrilov (was: Edson Tirelli) > Implement FEEL functions in preparation for DMN 1.2 > --------------------------------------------------- > > Key: DROOLS-2521 > URL: https://issues.jboss.org/browse/DROOLS-2521 > Project: Drools > Issue Type: Enhancement > Components: dmn engine > Affects Versions: 7.7.0.Final > Reporter: Edson Tirelli > Assignee: Fedor Gavrilov > Fix For: 7.8.0.Final > > Attachments: DMN12-190_proposal_v4.docx > > > There are a few new FEEL functions defined for DMN 1.2. > See attached document for the list of functions. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (DROOLS-2521) Implement FEEL functions in preparation for DMN 1.2

by Edson Tirelli (JIRA)

[ https://issues.jboss.org/browse/DROOLS-2521?page=com.atlassian.jira.plugi... ] Edson Tirelli updated DROOLS-2521: ---------------------------------- Attachment: DMN12-190_proposal_v4.docx > Implement FEEL functions in preparation for DMN 1.2 > --------------------------------------------------- > > Key: DROOLS-2521 > URL: https://issues.jboss.org/browse/DROOLS-2521 > Project: Drools > Issue Type: Enhancement > Components: dmn engine > Affects Versions: 7.7.0.Final > Reporter: Edson Tirelli > Assignee: Fedor Gavrilov > Fix For: 7.8.0.Final > > Attachments: DMN12-190_proposal_v4.docx > > > There are a few new FEEL functions defined for DMN 1.2. > See attached document for the list of functions. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (DROOLS-2521) Implement FEEL functions in preparation for DMN 1.2

by Edson Tirelli (JIRA)

Edson Tirelli created DROOLS-2521: ------------------------------------- Summary: Implement FEEL functions in preparation for DMN 1.2 Key: DROOLS-2521 URL: https://issues.jboss.org/browse/DROOLS-2521 Project: Drools Issue Type: Enhancement Components: dmn engine Affects Versions: 7.7.0.Final Reporter: Edson Tirelli Assignee: Edson Tirelli Fix For: 7.8.0.Final There are a few new FEEL functions defined for DMN 1.2. See attached document for the list of functions. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (WFLY-10314) Futures returned by CommandDispatcher.submitOnCluster(...) throw ClassCastException on get() if target node has no corresponding CommandDispatcher

by Radoslav Husar (JIRA)

[ https://issues.jboss.org/browse/WFLY-10314?page=com.atlassian.jira.plugin... ] Radoslav Husar updated WFLY-10314: ---------------------------------- Summary: Futures returned by CommandDispatcher.submitOnCluster(...) throw ClassCastException on get() if target node has no corresponding CommandDispatcher (was: Futures returned by CommandDispatcher.submitOnCluster(...) throws ClassCastException on get() if target node has no corresponding CommandDispatcher) > Futures returned by CommandDispatcher.submitOnCluster(...) throw ClassCastException on get() if target node has no corresponding CommandDispatcher > -------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: WFLY-10314 > URL: https://issues.jboss.org/browse/WFLY-10314 > Project: WildFly > Issue Type: Bug > Components: Clustering > Affects Versions: 12.0.0.Final > Reporter: Paul Ferraro > Assignee: Paul Ferraro > > The futures returned by CommandDispatcher.submitOnCluster(...) contain the command execution results per member of the cluster. However, if a given member has no active CommandDispatcher to receive the command, the resulting behavior is not handled, and the Future.get() will throw a ClassCastException. This is because the RspFilter contained in the RspOptions as passed to the MessageDispatcher is not invoked for unicast requests. > The API will likely need to be adjusted/documented to define what behavior to expect in this scenario. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (SWSQE-193) Move Ansible Files from Hawkular QE Utils to Kiali

by Guilherme Baufaker Rêgo (JIRA)

[ https://issues.jboss.org/browse/SWSQE-193?page=com.atlassian.jira.plugin.... ] Guilherme Baufaker Rêgo resolved SWSQE-193. ------------------------------------------- Resolution: Done > Move Ansible Files from Hawkular QE Utils to Kiali > -------------------------------------------------- > > Key: SWSQE-193 > URL: https://issues.jboss.org/browse/SWSQE-193 > Project: Kiali QE > Issue Type: Task > Reporter: Guilherme Baufaker Rêgo > Assignee: Guilherme Baufaker Rêgo > -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (WFCORE-3789) Unable to pass any attribute to jboss-cli.ps1 script

by Tomas Terem (JIRA)

Tomas Terem created WFCORE-3789: ----------------------------------- Summary: Unable to pass any attribute to jboss-cli.ps1 script Key: WFCORE-3789 URL: https://issues.jboss.org/browse/WFCORE-3789 Project: WildFly Core Issue Type: Bug Components: CLI, Scripts Reporter: Tomas Terem Assignee: Jean-Francois Denise Unable to pass any attribute to jboss-cli.ps1 script {noformat} C:\wildfly\wildfly-12.0.0.Final\bin> PowerShell -ExecutionPolicy Bypass -File jboss-cli.ps1 --help '--help-Dcom.ibm.jsse2.overrideDefaultTLS=true' is not a valid operation name. C:\wildfly\wildfly-12.0.0.Final\bin> PowerShell -ExecutionPolicy Bypass -File jboss-cli.ps1 --version '--version-Dcom.ibm.jsse2.overrideDefaultTLS=true' is not a valid operation name . C:\wildfly\wildfly-12.0.0.Final\bin> PowerShell -ExecutionPolicy Bypass -File jboss-cli.ps1 -c '-c-Dcom.ibm.jsse2.overrideDefaultTLS=true' is not a valid operation name. C:\wildfly\wildfly-12.0.0.Final\bin> PowerShell -ExecutionPolicy Bypass -File jboss-cli.ps1 You are disconnected at the moment. Type 'connect' to connect to the server or ' help' for the list of supported commands. [disconnected /] {noformat} I am using PowerShell 4.0 {noformat} PS C:\> $PSVersionTable.PSVersion Major Minor Build Revision ----- ----- ----- -------- 4 0 -1 -1 PS C:\> {noformat} This behaviour is caused by this change: https://github.com/wildfly/wildfly-core/blame/master/core-feature-pack/sr... -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

[JBoss JIRA] (WFLY-10317) TimeoutException caused by CIRCULAR REFERENCE in a 4 node cluster during fail-over

by tommaso borgato (JIRA)

[ https://issues.jboss.org/browse/WFLY-10317?page=com.atlassian.jira.plugin... ] tommaso borgato updated WFLY-10317: ----------------------------------- Description: We used master branch from [wildfly|https://github.com/wildfly/wildfly.git]. We observe lot of errors (more than a hundred on each node) like the following, note they occur right after one node has been failed and restarted and then another node is failed: {noformat} 2018/04/26 03:27:37:448 EDT [DEBUG][Thread-82] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - JBossStartup, server started! [JBossINF] [0m[0m03:28:39,359 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,366 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[0m03:28:39,366 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,367 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[0m03:28:39,368 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,369 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[0m03:28:39,369 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,370 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[31m03:28:49,087 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p10-t1) ISPN000136: Error executing command PrepareCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at org.infinispan.interceptors.impl.BaseStateTransferInterceptor$CancellableRetry.run(BaseStateTransferInterceptor.java:333) [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] [JBossINF] [0m[31m03:28:49,094 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-12) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:122) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.$closeResource(CacheRegistry.java:105) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.getEntry(CacheRegistry.java:178) [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanRouteLocator.locate(InfinispanRouteLocator.java:57) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionIdentifierCodec.encode(DistributableSessionIdentifierCodec.java:48) [JBossINF] at org.wildfly.extension.undertow.session.CodecSessionConfig.findSessionId(CodecSessionConfig.java:60) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl$ServletContextSessionConfig.findSessionId(ServletContextImpl.java:1215) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:158) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:858) [JBossINF] at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:412) [JBossINF] at org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) [JBossINF] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:241) [JBossINF] at org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:152) [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:246) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:291) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at org.infinispan.interceptors.impl.BaseStateTransferInterceptor$CancellableRetry.run(BaseStateTransferInterceptor.java:333) [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 more [JBossINF] Suppressed: java.util.concurrent.ExecutionException: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) [JBossINF] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) [JBossINF] at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:82) [JBossINF] at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:37) [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:122) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.$closeResource(CacheRegistry.java:105) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.getEntry(CacheRegistry.java:178) [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanRouteLocator.locate(InfinispanRouteLocator.java:57) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionIdentifierCodec.encode(DistributableSessionIdentifierCodec.java:48) [JBossINF] at org.wildfly.extension.undertow.session.CodecSessionConfig.findSessionId(CodecSessionConfig.java:60) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl$ServletContextSessionConfig.findSessionId(ServletContextImpl.java:1215) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:158) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:858) [JBossINF] at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:412) [JBossINF] at org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) [JBossINF] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:241) [JBossINF] at org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:152) [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:246) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:291) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] ... 1 more [JBossINF] [CIRCULAR REFERENCE:org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29] {noformat} We tried also with tag [12.0.0.Final|https://github.com/wildfly/wildfly/tree/12.0.0.Final], we did not have CIRCULAR REFERENCE, but we observed a high number (over a hundred for each node) of the following exceptions: {noformat} [JBossINF] [0m[33m11:29:05,812 WARN [org.infinispan.commons.tx.TransactionImpl] (default task-23) ISPN000927: exception while committing: javax.transaction.xa.XAException [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:222) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:168) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:126) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:446) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:493) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSession.$closeResource(DistributableSession.java:92) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:91) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) [JBossINF] at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:577) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:339) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) [JBossINF] ... 32 more [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 moreorg.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) [JBossINF] ... 32 more [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 more {noformat} was: We used master branch from [wildfly|https://github.com/wildfly/wildfly.git]. We observe lot of errors (more than a hundred on each node) like the following, note they occur right after one node has been failed and restarted and then another node is failed: {noformat} 2018/04/26 03:27:37:448 EDT [DEBUG][Thread-82] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - JBossStartup, server started! [JBossINF] [0m[0m03:28:39,359 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,366 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[0m03:28:39,366 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,367 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[0m03:28:39,368 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,369 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[0m03:28:39,369 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] [JBossINF] [0m[0m03:28:39,370 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster [JBossINF] [0m[31m03:28:49,087 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p10-t1) ISPN000136: Error executing command PrepareCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at org.infinispan.interceptors.impl.BaseStateTransferInterceptor$CancellableRetry.run(BaseStateTransferInterceptor.java:333) [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] [JBossINF] [0m[31m03:28:49,094 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-12) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:122) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.$closeResource(CacheRegistry.java:105) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.getEntry(CacheRegistry.java:178) [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanRouteLocator.locate(InfinispanRouteLocator.java:57) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionIdentifierCodec.encode(DistributableSessionIdentifierCodec.java:48) [JBossINF] at org.wildfly.extension.undertow.session.CodecSessionConfig.findSessionId(CodecSessionConfig.java:60) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl$ServletContextSessionConfig.findSessionId(ServletContextImpl.java:1215) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:158) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:858) [JBossINF] at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:412) [JBossINF] at org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) [JBossINF] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:241) [JBossINF] at org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:152) [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:246) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:291) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at org.infinispan.interceptors.impl.BaseStateTransferInterceptor$CancellableRetry.run(BaseStateTransferInterceptor.java:333) [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 more [JBossINF] Suppressed: java.util.concurrent.ExecutionException: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 [JBossINF] at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) [JBossINF] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) [JBossINF] at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:82) [JBossINF] at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:37) [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:122) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.$closeResource(CacheRegistry.java:105) [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.getEntry(CacheRegistry.java:178) [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanRouteLocator.locate(InfinispanRouteLocator.java:57) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionIdentifierCodec.encode(DistributableSessionIdentifierCodec.java:48) [JBossINF] at org.wildfly.extension.undertow.session.CodecSessionConfig.findSessionId(CodecSessionConfig.java:60) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl$ServletContextSessionConfig.findSessionId(ServletContextImpl.java:1215) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:158) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:858) [JBossINF] at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:412) [JBossINF] at org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) [JBossINF] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:241) [JBossINF] at org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:152) [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:246) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:291) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] ... 1 more [JBossINF] [CIRCULAR REFERENCE:org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29] {noformat} We tried also with tag version and with both versions, we observed a high number of the following exceptions: {noformat} [JBossINF] [0m[33m11:29:05,812 WARN [org.infinispan.commons.tx.TransactionImpl] (default task-23) ISPN000927: exception while committing: javax.transaction.xa.XAException [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:222) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:168) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:126) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:446) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:493) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSession.$closeResource(DistributableSession.java:92) [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:91) [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) [JBossINF] at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:577) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:339) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) [JBossINF] ... 32 more [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 moreorg.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) [JBossINF] ... 32 more [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 more {noformat} > TimeoutException caused by CIRCULAR REFERENCE in a 4 node cluster during fail-over > ---------------------------------------------------------------------------------- > > Key: WFLY-10317 > URL: https://issues.jboss.org/browse/WFLY-10317 > Project: WildFly > Issue Type: Bug > Components: Clustering > Affects Versions: 13.0.0.Beta1 > Reporter: tommaso borgato > Assignee: Paul Ferraro > Attachments: clusterbench-ee7.ear > > > We used master branch from [wildfly|https://github.com/wildfly/wildfly.git]. > We observe lot of errors (more than a hundred on each node) like the following, note they occur right after one node has been failed and restarted and then another node is failed: > {noformat} > 2018/04/26 03:27:37:448 EDT [DEBUG][Thread-82] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - JBossStartup, server started! > [JBossINF] [0m[0m03:28:39,359 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] > [JBossINF] [0m[0m03:28:39,366 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster > [JBossINF] [0m[0m03:28:39,366 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] > [JBossINF] [0m[0m03:28:39,367 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster > [JBossINF] [0m[0m03:28:39,368 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] > [JBossINF] [0m[0m03:28:39,369 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster > [JBossINF] [0m[0m03:28:39,369 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN000094: Received new cluster view for channel ejb: [perf20|6] (3) [perf20, perf21, perf18] > [JBossINF] [0m[0m03:28:39,370 INFO [org.infinispan.CLUSTER] (thread-22,ejb,perf18) ISPN100001: Node perf19 left the cluster > [JBossINF] [0m[31m03:28:49,087 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p10-t1) ISPN000136: Error executing command PrepareCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 > [JBossINF] at org.infinispan.interceptors.impl.BaseStateTransferInterceptor$CancellableRetry.run(BaseStateTransferInterceptor.java:333) > [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [JBossINF] at java.lang.Thread.run(Thread.java:748) > [JBossINF] > [JBossINF] [0m[31m03:28:49,094 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-12) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 > [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) > [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) > [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155) > [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:122) > [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) > [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) > [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.$closeResource(CacheRegistry.java:105) > [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.getEntry(CacheRegistry.java:178) > [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanRouteLocator.locate(InfinispanRouteLocator.java:57) > [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionIdentifierCodec.encode(DistributableSessionIdentifierCodec.java:48) > [JBossINF] at org.wildfly.extension.undertow.session.CodecSessionConfig.findSessionId(CodecSessionConfig.java:60) > [JBossINF] at io.undertow.servlet.spec.ServletContextImpl$ServletContextSessionConfig.findSessionId(ServletContextImpl.java:1215) > [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:158) > [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:858) > [JBossINF] at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:412) > [JBossINF] at org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) > [JBossINF] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:241) > [JBossINF] at org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:152) > [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:246) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:291) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) > [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) > [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) > [JBossINF] at java.lang.Thread.run(Thread.java:748) > [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 > [JBossINF] at org.infinispan.interceptors.impl.BaseStateTransferInterceptor$CancellableRetry.run(BaseStateTransferInterceptor.java:333) > [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [JBossINF] ... 1 more > [JBossINF] Suppressed: java.util.concurrent.ExecutionException: org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29 > [JBossINF] at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) > [JBossINF] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) > [JBossINF] at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:82) > [JBossINF] at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:37) > [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) > [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) > [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155) > [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:122) > [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) > [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) > [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.$closeResource(CacheRegistry.java:105) > [JBossINF] at org.wildfly.clustering.server.registry.CacheRegistry.getEntry(CacheRegistry.java:178) > [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanRouteLocator.locate(InfinispanRouteLocator.java:57) > [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionIdentifierCodec.encode(DistributableSessionIdentifierCodec.java:48) > [JBossINF] at org.wildfly.extension.undertow.session.CodecSessionConfig.findSessionId(CodecSessionConfig.java:60) > [JBossINF] at io.undertow.servlet.spec.ServletContextImpl$ServletContextSessionConfig.findSessionId(ServletContextImpl.java:1215) > [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:158) > [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:858) > [JBossINF] at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:412) > [JBossINF] at org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) > [JBossINF] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:241) > [JBossINF] at org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:152) > [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:246) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:291) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1521) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) > [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) > [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) > [JBossINF] ... 1 more > [JBossINF] [CIRCULAR REFERENCE:org.infinispan.util.concurrent.TimeoutException: Timed out waiting for topology 29] > {noformat} > We tried also with tag [12.0.0.Final|https://github.com/wildfly/wildfly/tree/12.0.0.Final], we did not have CIRCULAR REFERENCE, but we observed a high number (over a hundred for each node) of the following exceptions: > {noformat} > [JBossINF] [0m[33m11:29:05,812 WARN [org.infinispan.commons.tx.TransactionImpl] (default task-23) ISPN000927: exception while committing: javax.transaction.xa.XAException > [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:222) > [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:168) > [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:126) > [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:446) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:493) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) > [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) > [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) > [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSession.$closeResource(DistributableSession.java:92) > [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:91) > [JBossINF] at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) > [JBossINF] at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:577) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:339) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) > [JBossINF] at [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) > [JBossINF] at java.lang.Thread.run(Thread.java:748) > [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 > [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) > [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) > [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) > [JBossINF] ... 32 more > [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 > [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) > [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) > [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) > [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [JBossINF] ... 1 moreorg.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) > [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1526) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) > [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) > [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) > [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) > [JBossINF] at java.lang.Thread.run(Thread.java:748) > [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 > [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) > [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) > [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) > [JBossINF] ... 32 more > [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 49235 from perf20 > [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) > [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) > [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) > [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [JBossINF] ... 1 more > {noformat} -- This message was sent by Atlassian JIRA (v7.5.0#75005)

6 years, 1 month

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

jboss-jira May 2018