[JBoss JIRA] (WFLY-9892) Upgrade org.apache.santuario.xmlsec to 2.1.1. caused regression in PicketLinkSTS
by Jan Kalina (JIRA)
[ https://issues.jboss.org/browse/WFLY-9892?page=com.atlassian.jira.plugin.... ]
Jan Kalina commented on WFLY-9892:
----------------------------------
Maybe the issue is captured by PicketLink federation unit tests already - upgrade of {{org.apache.santuario}} in {{picketlink/modules/federation/pom.xml}} to {{2.1.1}} causes failures:
{code}
Tests in error:
SAMLAssertionWrappingAttackTestCase.testWrappingAttack » Unexpected exception...
SAMLAssertionWrappingAttackTestCase.testForceWrappingAttack:252 » Marshal Inva...
DomElementToStaxWritingTestCase.testDOM2Stax:48 » Parsing PL00074: Parsing Err...
{code}
> Upgrade org.apache.santuario.xmlsec to 2.1.1. caused regression in PicketLinkSTS
> --------------------------------------------------------------------------------
>
> Key: WFLY-9892
> URL: https://issues.jboss.org/browse/WFLY-9892
> Project: WildFly
> Issue Type: Bug
> Components: Security
> Affects Versions: 12.0.0.Beta1
> Reporter: Ondrej Lukas
> Priority: Blocker
> Attachments: ejb-security-picketlink.zip, ejb-test.jar, picketlink-sts.war, sts-config.properties
>
>
> When token from PicketLink STS is issued and signed then it is not able to be used for authentication through Remoting in WildFly 12 (i.e. it cannot be set as {{remote.connection.main.password}} property which can be used in PicketLink {{org.picketlink.identity.federation.bindings.jboss.auth.SAML2STSLoginModule}}). It seems it is caused by upgrade of org.apache.santuario.xmlsec to version 2.1.1. [1]. When WILDFLY11_HOME/modules/system/layers/base/org/apache/santuario/xmlsec/main/xmlsec-2.0.8.jar is placed to WildFly 12 modules then it works correctly.
> We report it as a blocker since it is regression - application which works correctly on WildFly 11 stops to work on WildFly 12 - users are not able to authenticate through Remoting with signed tokens from PicketLink STS correctly.
> Remoting fails due to following exception:
> {code}
> java.lang.IllegalArgumentException: ELY05131: Invalid ASCII control "0xA"
> at org.wildfly.security.sasl.util.StringPrep.forbidAsciiControl(StringPrep.java:117)
> at org.wildfly.security.sasl.util.StringPrep.encode(StringPrep.java:295)
> at org.wildfly.security.sasl.util.StringPrep.encode(StringPrep.java:196)
> at org.wildfly.security.sasl.plain.PlainSaslClient.evaluateChallenge(PlainSaslClient.java:95)
> at org.wildfly.security.sasl.util.AbstractDelegatingSaslClient.evaluateChallenge(AbstractDelegatingSaslClient.java:54)
> at org.wildfly.security.sasl.util.PrivilegedSaslClient.lambda$evaluateChallenge$0(PrivilegedSaslClient.java:55)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.wildfly.security.sasl.util.PrivilegedSaslClient.evaluateChallenge(PrivilegedSaslClient.java:55)
> at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.lambda$handleEvent$1(ClientConnectionOpenListener.java:460)
> at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:926)
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1979)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1481)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1374)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> It is caused by different formating value of SignatureValue in assertion. In WildFly 11 SignatureValue looks like:
> {code}
> <dsig:SignatureValue xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">nFVkKrXTyYEQ9cwc9OOgySYebEtwzw4alVYP0viXzvqZAUAKtAXEBAfDB8xIOms78twlDdq79MiSvk8OrOdf126Kw/IR8JRn1fYyZ5tsIRcNoTXMgGaTqhrn/HKlLqqqHhVHrJURunqkSzTTxylA2AEPhEDD5Y7hS0W2ZZCeSvuri+PRDSTrRnuedz0yQuHQu1mZ0gjoEFbHh4Wkkn5Ac1R4gmewmmzPud+ZE6Ux4YpeHzQ8rAvZ4bDk6j+eQIRsSxFTLo5RSA3FWN8+lUNV/CSRqBPXsK7QxOaTdBgF+4NXWeExrNJ9SeVFcf9yelvReAtR2JNZ6DUY8u45KtXmLw==</dsig:SignatureValue>
> {code}
> In WildFly 12 it looks like (there are end of lines):
> {code}
> <dsig:SignatureValue xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">cUNpFJIZlLYrBDZtQSTDrq2K6PbnAHyg2qbx/D5FuB4XMjdQ5oxQjkMejLyelnA7s4GFusoLhahl
> qlTOT8UrOyxrR4yYAmJ/e5s+f4gys926+tbiraT/3/wG8wM/Lvcjvk5Ap69zODuRYpypsWfA4jrI
> 7TTBXVPGy8g4KUdnFviUiTuFTc2Ghgxp53AmUuLis/THyP28jE7+28//q8bi/bQrFwHC6tWX67+N
> K1duFCOcQ6IPIKeVrePZz55Ivgl+WWdkF6uYCz5IdMzurhzmeQ3K8DAMIxz/MG67VWJIOnuGNWF7
> nmdye5zd9AFcRsr1XadvZJCbGNfuc89AL5inCg==</dsig:SignatureValue>
> {code}
> [1] https://github.com/wildfly/wildfly/commit/536de514829f2187abf1126c8916a04...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (WFCORE-3647) CLI multi page output does not work correctly on Cygwin
by Erich Duda (JIRA)
Erich Duda created WFCORE-3647:
----------------------------------
Summary: CLI multi page output does not work correctly on Cygwin
Key: WFCORE-3647
URL: https://issues.jboss.org/browse/WFCORE-3647
Project: WildFly Core
Issue Type: Bug
Components: CLI
Affects Versions: 3.1.0.Final
Reporter: Erich Duda
Assignee: Jean-Francois Denise
The Auto more/less in CLI for multi-page output (WFCORE-426) does not work correctly on Cygwin. Each key has to be pushed twice to interact.
It was hit on following Cygwin version.
{code}
$ uname -a
CYGWIN_NT-6.3 WIN-VSOI9S1FO1Q 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (WFLY-9888) Server log messages does not reflect the use of "distinct" naming for EJB's
by Chao Wang (JIRA)
[ https://issues.jboss.org/browse/WFLY-9888?page=com.atlassian.jira.plugin.... ]
Chao Wang commented on WFLY-9888:
---------------------------------
https://github.com/soul2zimate/quickstart/commits/WFLY-9888 example to see the actual JNDI mismatch with distinct-name defined
{code:java}
ejb:<appName>/<moduleName>/<distinctName>/<beanName>!<viewClassName>
{code}
> Server log messages does not reflect the use of "distinct" naming for EJB's
> ---------------------------------------------------------------------------
>
> Key: WFLY-9888
> URL: https://issues.jboss.org/browse/WFLY-9888
> Project: WildFly
> Issue Type: Enhancement
> Components: EJB
> Reporter: Wolf-Dieter Fink
> Assignee: Chao Wang
> Priority: Minor
>
> If an application (jboss-ejb3.xml) or server configuration is marked with the distinct flag
> <jboss:ejb-jar .... version="3.1" impl-version="2.0">
> <distinct-name>WOLF</distinct-name>
> </jboss:ejb-jar>
> The client need to use the the correct lookup with distinct name
> "ejb:MyEAR/MyEjbJar/WOLF/MyBean!MyInterface
> to get the bean proxy.
> Unfortunately there is no hint whether an application is marked with this distinct flag.
> The server log still show the same output as without like this:
> INFO WFLYEJB0473: JNDI bindings for session bean named 'SimpleBean' in deployment unit 'subdeployment "ejb.jar" of deployment "EAP71-PLAYGROUND-server.ear"' are as follows:
> java:global/EAP71-PLAYGROUND-server/ejb/SimpleBean!org.jboss.wfink.eap71.playground.Simple
> It would be good if the log message show the correct JNDI name or a hint that the EJB has a distinct name applied.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (DROOLS-2349) [DMN Editor] Decision Navigator dock
by Jozef Marko (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2349?page=com.atlassian.jira.plugi... ]
Jozef Marko commented on DROOLS-2349:
-------------------------------------
Could we please split this task into more PRs?
# Introduce just new view on left side
# Show Tree structure for DRG
# Show Tree Structure for selected element of DRG
## Input
## Decision
## BKM
## ...
# Show Preview of selected item
Please let me know what do you think.
> [DMN Editor] Decision Navigator dock
> ------------------------------------
>
> Key: DROOLS-2349
> URL: https://issues.jboss.org/browse/DROOLS-2349
> Project: Drools
> Issue Type: Task
> Components: DMN Editor
> Reporter: Guilherme Carreiro
> Assignee: Guilherme Carreiro
> Attachments: prototype.png
>
>
> The "Decision Navigator" is docked next to the Project Explorer. See:
> !prototype.png|thumbnail!
> - Tree navigation offers a view of the entire DRG. Diagram nodes are represented within the tree.
> - User defined DRD’s are represented as top level tree nodes, with supporting decision and input data represented as subordinate tree nodes.
> - Tree structure only goes as deep as primary logic definition, such as a decision table (nested logic is not represented in the tree.)
> - The Navigator includes a preview image which represents a thumbnail view of the diagram view selected.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (DROOLS-2349) [DMN Editor] Decision Navigator dock
by Jozef Marko (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2349?page=com.atlassian.jira.plugi... ]
Jozef Marko updated DROOLS-2349:
--------------------------------
Component/s: DMN Editor
> [DMN Editor] Decision Navigator dock
> ------------------------------------
>
> Key: DROOLS-2349
> URL: https://issues.jboss.org/browse/DROOLS-2349
> Project: Drools
> Issue Type: Task
> Components: DMN Editor
> Reporter: Guilherme Carreiro
> Assignee: Guilherme Carreiro
> Attachments: prototype.png
>
>
> The "Decision Navigator" is docked next to the Project Explorer. See:
> !prototype.png|thumbnail!
> - Tree navigation offers a view of the entire DRG. Diagram nodes are represented within the tree.
> - User defined DRD’s are represented as top level tree nodes, with supporting decision and input data represented as subordinate tree nodes.
> - Tree structure only goes as deep as primary logic definition, such as a decision table (nested logic is not represented in the tree.)
> - The Navigator includes a preview image which represents a thumbnail view of the diagram view selected.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (DROOLS-2349) [DMN Editor] Decision Navigator dock
by Jozef Marko (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2349?page=com.atlassian.jira.plugi... ]
Jozef Marko updated DROOLS-2349:
--------------------------------
Tester: Jozef Marko
> [DMN Editor] Decision Navigator dock
> ------------------------------------
>
> Key: DROOLS-2349
> URL: https://issues.jboss.org/browse/DROOLS-2349
> Project: Drools
> Issue Type: Task
> Reporter: Guilherme Carreiro
> Assignee: Guilherme Carreiro
> Attachments: prototype.png
>
>
> The "Decision Navigator" is docked next to the Project Explorer. See:
> !prototype.png|thumbnail!
> - Tree navigation offers a view of the entire DRG. Diagram nodes are represented within the tree.
> - User defined DRD’s are represented as top level tree nodes, with supporting decision and input data represented as subordinate tree nodes.
> - Tree structure only goes as deep as primary logic definition, such as a decision table (nested logic is not represented in the tree.)
> - The Navigator includes a preview image which represents a thumbnail view of the diagram view selected.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months
[JBoss JIRA] (WFLY-9890) Timed out waiting for responses for request X from node Y immediately after node Y rejoins the cluster (failover)
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-9890?page=com.atlassian.jira.plugin.... ]
Michal Vinkler updated WFLY-9890:
---------------------------------
Description:
Seen in SF failover tests - scenarios:
failover-http-granular-shutdown-repl-sync (replication granularity is set to ATTRIBUTE level)
failover-http-session-jvmkill-repl-sync (replication granularity is set to SESSION level)
Description: In the failover test, each server is shut down and after some time it rejoins the cluster. After joining the cluster again, the load is distributed to this node. It seems that first request for some session handled by that node produces ERROR in the log of some other server until the time another cluster topology change occurs.
Stacktrace of the error:
{code}
[JBossINF] [0m[31m10:18:00,256 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p12-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionCreationMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAccessMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAttributeKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9[1])]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 30250 from perf18
[JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
{code}
Client gets the stale data:
{code}
2018/02/21 10:18:00:272 EST [WARN ][Runner - 16] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16>
org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16
at org.jboss.smartfrog.loaddriver.http.AbstractSerialNumberValidatorFactoryImpl$SerialNumberValidator.processRequest(AbstractSerialNumberValidatorFactoryImpl.java:133)
at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
at java.lang.Thread.run(Thread.java:748)
{code}
Example - timeline:
{code}
4 servers are running, 2000 clients, no issue in the beggining
2018/02/21 10:16:43:234 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 0 (perf18) is down.
- still no issues
2018/02/21 10:17:43:234 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Bringing back node 0 (perf18)
2018/02/21 10:17:55:611 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 0 (perf18) is up.
2018/02/21 10:18:00:272 EST [WARN ][Runner - 16] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16>
{code}
perf20 started logging the errors meanwhile and stopped only after perf19 was shut down:
{code}
[JBossINF] [0m[0m10:17:50,227 INFO [org.infinispan.CLUSTER] (thread-19,ejb,perf20) ISPN000094: Received new cluster view for channel ejb: [perf19|5] (4) [perf19, perf20, perf21, perf18]
[JBossINF] [0m[0m10:17:50,228 INFO [org.infinispan.CLUSTER] (thread-19,ejb,perf20) ISPN100000: Node perf18 joined the cluster
[JBossINF] [0m[31m10:18:00,256 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p12-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionCreationMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAccessMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAttributeKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9[1])]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 30250 from perf18
[JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF]
{code}
There were no issues in this scenario in EAP 7.1.0.GA.
Links:
Clients: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
perf18: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
perf20: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
Another test:
perf19: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
was:
Seen in SF failover tests - scenario: failover-http-granular-shutdown-repl-sync
(replication granularity is set to ATTRIBUTE level)
Description: In the failover test, each server is shut down and after some time it rejoins the cluster. After joining the cluster again, the load is distributed to this node. It seems that first request for some session handled by that node produces ERROR in the log of some other server until the time another cluster topology change occurs.
Stacktrace of the error:
{code}
[JBossINF] [0m[31m10:18:00,256 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p12-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionCreationMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAccessMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAttributeKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9[1])]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 30250 from perf18
[JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
{code}
Client gets the stale data:
{code}
2018/02/21 10:18:00:272 EST [WARN ][Runner - 16] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16>
org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16
at org.jboss.smartfrog.loaddriver.http.AbstractSerialNumberValidatorFactoryImpl$SerialNumberValidator.processRequest(AbstractSerialNumberValidatorFactoryImpl.java:133)
at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
at java.lang.Thread.run(Thread.java:748)
{code}
Example - timeline:
{code}
4 servers are running, 2000 clients, no issue in the beggining
2018/02/21 10:16:43:234 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 0 (perf18) is down.
- still no issues
2018/02/21 10:17:43:234 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Bringing back node 0 (perf18)
2018/02/21 10:17:55:611 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 0 (perf18) is up.
2018/02/21 10:18:00:272 EST [WARN ][Runner - 16] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16>
{code}
perf20 started logging the errors meanwhile and stopped only after perf19 was shut down:
{code}
[JBossINF] [0m[0m10:17:50,227 INFO [org.infinispan.CLUSTER] (thread-19,ejb,perf20) ISPN000094: Received new cluster view for channel ejb: [perf19|5] (4) [perf19, perf20, perf21, perf18]
[JBossINF] [0m[0m10:17:50,228 INFO [org.infinispan.CLUSTER] (thread-19,ejb,perf20) ISPN100000: Node perf18 joined the cluster
[JBossINF] [0m[31m10:18:00,256 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p12-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionCreationMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAccessMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAttributeKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9[1])]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 30250 from perf18
[JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF]
{code}
There were no issues in this scenario in EAP 7.1.0.GA.
Links:
Clients: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
perf18: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
perf20: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
> Timed out waiting for responses for request X from node Y immediately after node Y rejoins the cluster (failover)
> -----------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-9890
> URL: https://issues.jboss.org/browse/WFLY-9890
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 12.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
> Priority: Critical
>
> Seen in SF failover tests - scenarios:
> failover-http-granular-shutdown-repl-sync (replication granularity is set to ATTRIBUTE level)
> failover-http-session-jvmkill-repl-sync (replication granularity is set to SESSION level)
> Description: In the failover test, each server is shut down and after some time it rejoins the cluster. After joining the cluster again, the load is distributed to this node. It seems that first request for some session handled by that node produces ERROR in the log of some other server until the time another cluster topology change occurs.
> Stacktrace of the error:
> {code}
> [JBossINF] [0m[31m10:18:00,256 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p12-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionCreationMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAccessMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAttributeKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9[1])]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 30250 from perf18
> [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163)
> [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86)
> [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21)
> [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> {code}
> Client gets the stale data:
> {code}
> 2018/02/21 10:18:00:272 EST [WARN ][Runner - 16] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16>
> org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16
> at org.jboss.smartfrog.loaddriver.http.AbstractSerialNumberValidatorFactoryImpl$SerialNumberValidator.processRequest(AbstractSerialNumberValidatorFactoryImpl.java:133)
> at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
> at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Example - timeline:
> {code}
> 4 servers are running, 2000 clients, no issue in the beggining
> 2018/02/21 10:16:43:234 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 0 (perf18) is down.
> - still no issues
> 2018/02/21 10:17:43:234 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Bringing back node 0 (perf18)
> 2018/02/21 10:17:55:611 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 0 (perf18) is up.
> 2018/02/21 10:18:00:272 EST [WARN ][Runner - 16] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 60, received 59, Runner: 16>
> {code}
> perf20 started logging the errors meanwhile and stopped only after perf19 was shut down:
> {code}
> [JBossINF] [0m[0m10:17:50,227 INFO [org.infinispan.CLUSTER] (thread-19,ejb,perf20) ISPN000094: Received new cluster view for channel ejb: [perf19|5] (4) [perf19, perf20, perf21, perf18]
> [JBossINF] [0m[0m10:17:50,228 INFO [org.infinispan.CLUSTER] (thread-19,ejb,perf20) ISPN100000: Node perf18 joined the cluster
> [JBossINF] [0m[31m10:18:00,256 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p12-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionCreationMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAccessMetaDataKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9), SessionAttributeKey(f25_wwXXpvtNC9QTHZUtSwhmEUmg-wI77RL_17b9[1])]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 30250 from perf18
> [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163)
> [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86)
> [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21)
> [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> [JBossINF]
> {code}
> There were no issues in this scenario in EAP 7.1.0.GA.
> Links:
> Clients: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
> perf18: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
> perf20: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
> Another test:
> perf19: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 7 months