[JBoss JIRA] (WFWIP-257) SSO layer cannot be properly used with SSO template
by Michal Jurc (Jira)
[ https://issues.jboss.org/browse/WFWIP-257?page=com.atlassian.jira.plugin.... ]
Michal Jurc updated WFWIP-257:
------------------------------
Steps to Reproduce:
Build from template:
1. Checkout and replace templates for the stream:
{code:bash}
for resource in \
eap-cd-image-stream.json \
eap-cd-amq-persistent-s2i.json \
eap-cd-amq-s2i.json \
eap-cd-basic-s2i.json \
eap-cd-https-s2i.json \
eap-cd-sso-s2i.json \
eap-cd-starter-s2i.json \
eap-cd-third-party-db-s2i.json \
eap-cd-tx-recovery-s2i.json
do
oc replace --force -f \
https://raw.githubusercontent.com/jboss-container-images/jboss-eap-7-open...
done
{code}
2. Initiate the application build (replace {{<NAMESPACE_NAME>}} with your namespace and {{<OPENSHIFT_HOST>}} with your OpenShift host ip):
{code:bash}oc new-app --template=eap-cd-sso-s2i -p SSO_REALM=xpaas -p HTTPS_KEYSTORE=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io.keystore -p SSO_SAML_KEYSTORE_SECRET=eap-app-secret -p SSO_SAML_KEYSTORE_PASSWORD=password -p SSO_SERVICE_URL= -p SSO_TRUSTSTORE=truststore -p HOSTNAME_HTTPS=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io -p JGROUPS_ENCRYPT_KEYSTORE=jgroups.jceks -p HTTPS_PASSWORD=password -p SSO_SAML_CERTIFICATE_NAME=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io -p HTTPS_NAME=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io -p JGROUPS_ENCRYPT_PASSWORD=xpaasQEpassword -p JGROUPS_ENCRYPT_SECRET=eap-app-secret -p SSO_URL=https://secure-sso-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io/auth -p SSO_TRUSTSTORE_SECRET=eap-app-secret -p SSO_PASSWORD=creator -p SSO_SAML_KEYSTORE=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io.keystore -p HTTPS_SECRET=eap-app-secret -p JGROUPS_ENCRYPT_NAME=secret-key -p SSO_USERNAME=client -p APPLICATION_NAME=eap-app -p IMAGE_STREAM_NAMESPACE=<NAMESPACE_NAME> -p SSO_TRUSTSTORE_PASSWORD=password GALLEON_PROVISION_LAYERS=datasources-web-server,sso
{code}
3. Check the resulting build in the namespace.
was:
Build from template:
1. Checkout and replace templates for the stream:
{code:bash}
for resource in \
eap-cd-image-stream.json \
eap-cd-amq-persistent-s2i.json \
eap-cd-amq-s2i.json \
eap-cd-basic-s2i.json \
eap-cd-https-s2i.json \
eap-cd-sso-s2i.json \
eap-cd-starter-s2i.json \
eap-cd-third-party-db-s2i.json \
eap-cd-tx-recovery-s2i.json
do
oc replace --force -f \
https://raw.githubusercontent.com/jboss-container-images/jboss-eap-7-open...
done
{code}
2. Initiate the application build (replace {{<NAMESPACE_NAME>}} with your namespace and {{<OPENSHIFT_HOST>}} with your OpenShift host ip):
{code:bash}oc new-app --template=eap72-openjdk11-sso-s2i -p SSO_REALM=xpaas -p HTTPS_KEYSTORE=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io.keystore -p SSO_SAML_KEYSTORE_SECRET=eap-app-secret -p SSO_SAML_KEYSTORE_PASSWORD=password -p SSO_SERVICE_URL= -p SSO_TRUSTSTORE=truststore -p HOSTNAME_HTTPS=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io -p JGROUPS_ENCRYPT_KEYSTORE=jgroups.jceks -p HTTPS_PASSWORD=password -p SSO_SAML_CERTIFICATE_NAME=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io -p HTTPS_NAME=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io -p JGROUPS_ENCRYPT_PASSWORD=xpaasQEpassword -p JGROUPS_ENCRYPT_SECRET=eap-app-secret -p SSO_URL=https://secure-sso-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io/auth -p SSO_TRUSTSTORE_SECRET=eap-app-secret -p SSO_PASSWORD=creator -p SSO_SAML_KEYSTORE=secure-eap-app-<NAMESPACE_NAME>.<OPENSHIFT_HOST>.nip.io.keystore -p HTTPS_SECRET=eap-app-secret -p JGROUPS_ENCRYPT_NAME=secret-key -p SSO_USERNAME=client -p APPLICATION_NAME=eap-app -p IMAGE_STREAM_NAMESPACE=<NAMESPACE_NAME> -p SSO_TRUSTSTORE_PASSWORD=password GALLEON_PROVISION_LAYERS=datasources-web-server,sso
{code}
3. Check the resulting build in the namespace.
> SSO layer cannot be properly used with SSO template
> ----------------------------------------------------
>
> Key: WFWIP-257
> URL: https://issues.jboss.org/browse/WFWIP-257
> Project: WildFly WIP
> Issue Type: Bug
> Components: OpenShift
> Reporter: Michal Jurc
> Assignee: Brian Stansberry
> Priority: Blocker
>
> Currently available OpenShift templates for EAP CD are not able to properly configure EAP server with SSO client, resulting in the following errors:
> {code}15:55:01,843 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 38) MSC000001: Failed to start service jboss.deployment.unit."app-profile-jsp.war".undertow-deployment: org.jboss.msc.service.StartException in service jboss.deployment.unit."app-profile-jsp.war".undertow-deployment: java.lang.RuntimeException: java.lang.IllegalStateException: The required mechanism 'KEYCLOAK' is not available in mechanisms [BASIC, CLIENT_CERT, DIGEST, FORM] from the HttpAuthenticationFactory.
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
> at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
> at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
> at java.base/java.lang.Thread.run(Thread.java:834)
> at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.JBossThread.run(JBossThread.java:485)
> Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: The required mechanism 'KEYCLOAK' is not available in mechanisms [BASIC, CLIENT_CERT, DIGEST, FORM] from the HttpAuthenticationFactory.
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:254)
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:96)
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
> ... 8 more
> Caused by: java.lang.IllegalStateException: The required mechanism 'KEYCLOAK' is not available in mechanisms [BASIC, CLIENT_CERT, DIGEST, FORM] from the HttpAuthenticationFactory.
> at org.wildfly.security.elytron-web.undertow-server-servlet@1.6.0.Final-redhat-00001//org.wildfly.elytron.web.undertow.server.servlet.AuthenticationManager.initialSecurityHandler(AuthenticationManager.java:153)
> at org.wildfly.security.elytron-web.undertow-server-servlet@1.6.0.Final-redhat-00001//org.wildfly.elytron.web.undertow.server.servlet.AuthenticationManager.lambda$configure$2(AuthenticationManager.java:98)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.setupSecurityHandlers(DeploymentManagerImpl.java:442)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.access$600(DeploymentManagerImpl.java:121)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:224)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:186)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
> at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
> at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:252)
> ... 10 more{code}
> I am marking this issue as blocker since the template is currently the only documented way for EAP to use SSO client on OpenShift.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (WFWIP-257) SSO layer cannot be properly used with SSO template
by Michal Jurc (Jira)
Michal Jurc created WFWIP-257:
---------------------------------
Summary: SSO layer cannot be properly used with SSO template
Key: WFWIP-257
URL: https://issues.jboss.org/browse/WFWIP-257
Project: WildFly WIP
Issue Type: Bug
Components: OpenShift
Reporter: Michal Jurc
Assignee: Brian Stansberry
Currently available OpenShift templates for EAP CD are not able to properly configure EAP server with SSO client, resulting in the following errors:
{code}15:55:01,843 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 38) MSC000001: Failed to start service jboss.deployment.unit."app-profile-jsp.war".undertow-deployment: org.jboss.msc.service.StartException in service jboss.deployment.unit."app-profile-jsp.war".undertow-deployment: java.lang.RuntimeException: java.lang.IllegalStateException: The required mechanism 'KEYCLOAK' is not available in mechanisms [BASIC, CLIENT_CERT, DIGEST, FORM] from the HttpAuthenticationFactory.
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.base/java.lang.Thread.run(Thread.java:834)
at org.jboss.threads@2.3.3.Final-redhat-00001//org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: The required mechanism 'KEYCLOAK' is not available in mechanisms [BASIC, CLIENT_CERT, DIGEST, FORM] from the HttpAuthenticationFactory.
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:254)
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:96)
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
... 8 more
Caused by: java.lang.IllegalStateException: The required mechanism 'KEYCLOAK' is not available in mechanisms [BASIC, CLIENT_CERT, DIGEST, FORM] from the HttpAuthenticationFactory.
at org.wildfly.security.elytron-web.undertow-server-servlet@1.6.0.Final-redhat-00001//org.wildfly.elytron.web.undertow.server.servlet.AuthenticationManager.initialSecurityHandler(AuthenticationManager.java:153)
at org.wildfly.security.elytron-web.undertow-server-servlet@1.6.0.Final-redhat-00001//org.wildfly.elytron.web.undertow.server.servlet.AuthenticationManager.lambda$configure$2(AuthenticationManager.java:98)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.setupSecurityHandlers(DeploymentManagerImpl.java:442)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.access$600(DeploymentManagerImpl.java:121)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:224)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:186)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
at org.wildfly.extension.undertow@7.3.0.CD18-redhat-00002//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504)
at io.undertow.servlet@2.0.26.SP3-redhat-00001//io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:252)
... 10 more{code}
I am marking this issue as blocker since the template is currently the only documented way for EAP to use SSO client on OpenShift.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (JGRP-2396) increasing networkdata, cpu and heap
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2396?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2396:
--------------------------------
The attachments don't show anything out of the ordinary; I suggest attach Flight Recorder during an idle phase with high CPU usage, and attach the JFR to this case.
> increasing networkdata, cpu and heap
> ------------------------------------
>
> Key: JGRP-2396
> URL: https://issues.jboss.org/browse/JGRP-2396
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.0.19
> Reporter: Rob van der Boom
> Assignee: Bela Ban
> Priority: Major
> Attachments: Schermafbeelding 2019-11-08 om 15.43.04.png, Schermafbeelding 2019-11-08 om 15.44.52.png, Schermafbeelding 2019-11-08 om 16.00.48.png, jstack-production-pod0.dump, standalone-ha.xml
>
>
> hey,
> we have an keycloak (sso) setup, version 7.0.1 running in kubernetes - aws.
> Its build on wildfly 17, infinispan 9.4 and jgroups 4.0.19.
> We have 3 pods running in standalone-ha with cache setup on distribution (all 3 nodes - so equivalent to replication)
> ISSUE:
> We see a slowly growing of networkstatistics, heap and cpu, while the number of sessions in keycloak (cached) remain almost stable.
> The cpu growth is caused by the TQbundler process, which explaines the networkdata growth. It looks like this is causing also a memory leakage..
> every 5 days we have to restart the pods and then every resets to a very low level including the heap. this while all sessions are still valid and cached.
> The only issue i could find maybe related to this is:
> https://issues.jboss.org/browse/JGRP-2382?jql=project%20%3D%20JGRP%20AND%...
> Could this be the same issue and does it also cause increasing network and cpu (since that is why we have to restart, the heap has much space left !).
> And if so how does this issue continue since for us its a major issue.
> We als had this issue already in keycloak 5 (wildfly 15), thats why we upgraded to the latest available version.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (JGRP-2396) increasing networkdata, cpu and heap
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2396?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2396:
--------------------------------
No: the TransferQueueBundler is idle (WAITING).
> increasing networkdata, cpu and heap
> ------------------------------------
>
> Key: JGRP-2396
> URL: https://issues.jboss.org/browse/JGRP-2396
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.0.19
> Reporter: Rob van der Boom
> Assignee: Bela Ban
> Priority: Major
> Attachments: Schermafbeelding 2019-11-08 om 15.43.04.png, Schermafbeelding 2019-11-08 om 15.44.52.png, Schermafbeelding 2019-11-08 om 16.00.48.png, jstack-production-pod0.dump, standalone-ha.xml
>
>
> hey,
> we have an keycloak (sso) setup, version 7.0.1 running in kubernetes - aws.
> Its build on wildfly 17, infinispan 9.4 and jgroups 4.0.19.
> We have 3 pods running in standalone-ha with cache setup on distribution (all 3 nodes - so equivalent to replication)
> ISSUE:
> We see a slowly growing of networkstatistics, heap and cpu, while the number of sessions in keycloak (cached) remain almost stable.
> The cpu growth is caused by the TQbundler process, which explaines the networkdata growth. It looks like this is causing also a memory leakage..
> every 5 days we have to restart the pods and then every resets to a very low level including the heap. this while all sessions are still valid and cached.
> The only issue i could find maybe related to this is:
> https://issues.jboss.org/browse/JGRP-2382?jql=project%20%3D%20JGRP%20AND%...
> Could this be the same issue and does it also cause increasing network and cpu (since that is why we have to restart, the heap has much space left !).
> And if so how does this issue continue since for us its a major issue.
> We als had this issue already in keycloak 5 (wildfly 15), thats why we upgraded to the latest available version.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (JGRP-2395) LOCAL_PING fails when 2 nodes start at the same time
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2395?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2395:
--------------------------------
I added an item to the F2F agenda
> LOCAL_PING fails when 2 nodes start at the same time
> ----------------------------------------------------
>
> Key: JGRP-2395
> URL: https://issues.jboss.org/browse/JGRP-2395
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.1.6
> Reporter: Dan Berindei
> Assignee: Bela Ban
> Priority: Major
> Fix For: 4.1.8
>
>
> We have a test that starts 2 nodes in parallel ({{ConcurrentStartTest}} and it is randomly failing since we started using {{LOCAL_PING}}.
> {noformat}
> 01:02:11,930 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: discovery took 3 ms, members: 1 rsps (0 coords) [done]
> 01:02:11,930 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: discovery took 3 ms, members: 1 rsps (0 coords) [done]
> 01:02:11,931 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,931 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,931 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> 01:02:11,931 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> 01:02:11,931 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: I (Test-NodeB-43694) am the first of the nodes, will become coordinator
> 01:02:11,931 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: I (Test-NodeA-29550) am not the first of the nodes, waiting for another client to become coordinator
> 01:02:11,932 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: discovery took 0 ms, members: 1 rsps (0 coords) [done]
> 01:02:11,932 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,932 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> ...
> 01:02:11,941 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,941 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> 01:02:11,941 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: I (Test-NodeA-29550) am not the first of the nodes, waiting for another client to become coordinator
> 01:02:11,942 WARN (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: too many JOIN attempts (10): becoming singleton
> 01:02:11,942 DEBUG (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: installing view [Test-NodeA-29550|0] (1) [Test-NodeA-29550]
> 01:02:11,977 DEBUG (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: created cluster (first member). My view is [Test-NodeB-43694|0], impl is org.jgroups.protocols.pbcast.CoordGmsImpl
> 01:02:11,977 DEBUG (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: created cluster (first member). My view is [Test-NodeA-29550|0], impl is org.jgroups.protocols.pbcast.CoordGmsImpl
> {noformat}
> The problem seems to be that it takes longer for the coordinator to install the initial view and update {{LOCAL_PING}}'s {{PingData}} then it takes the other node to retry the discovery process 10 times.
> In some cases there is no retry, because one node starts slightly faster, but it's not yet coordinator when the 2nd node does its discovery, and both nodes decide they should be coordinator:
> {noformat}
> 01:13:44,460 INFO (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-5386: no members discovered after 3 ms: creating cluster as first member
> 01:13:44,463 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: discovery took 1 ms, members: 1 rsps (0 coords) [done]
> 01:13:44,465 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:13:44,465 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: nodes to choose new coord from are: [Test-NodeB-51165, Test-NodeA-5386]
> 01:13:44,466 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: I (Test-NodeB-51165) am the first of the nodes, will become coordinator
> 01:13:44,466 DEBUG (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: installing view [Test-NodeB-51165|0] (1) [Test-NodeB-51165]
> 01:13:44,466 DEBUG (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-5386: installing view [Test-NodeA-5386|0] (1) [Test-NodeA-5386]
> {noformat}
> This second failure mode seems to go away if I move the {{discovery}} map access inside the {{synchronized}} block both in {{findMembers()}} and in {{down()}}.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (JGRP-2395) LOCAL_PING fails when 2 nodes start at the same time
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2395?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2395:
--------------------------------
I commented on JGRP-2398.
Re applying this behavior to regular discovery protocols: I think it depends on thread timing; when multiple threads are started at the same time on different host, it is hard to avoid merging, as some nodes will not find any others, others will find different sets etc.
Let's discuss at the F2F.
> LOCAL_PING fails when 2 nodes start at the same time
> ----------------------------------------------------
>
> Key: JGRP-2395
> URL: https://issues.jboss.org/browse/JGRP-2395
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.1.6
> Reporter: Dan Berindei
> Assignee: Bela Ban
> Priority: Major
> Fix For: 4.1.8
>
>
> We have a test that starts 2 nodes in parallel ({{ConcurrentStartTest}} and it is randomly failing since we started using {{LOCAL_PING}}.
> {noformat}
> 01:02:11,930 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: discovery took 3 ms, members: 1 rsps (0 coords) [done]
> 01:02:11,930 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: discovery took 3 ms, members: 1 rsps (0 coords) [done]
> 01:02:11,931 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,931 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,931 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> 01:02:11,931 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> 01:02:11,931 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: I (Test-NodeB-43694) am the first of the nodes, will become coordinator
> 01:02:11,931 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: I (Test-NodeA-29550) am not the first of the nodes, waiting for another client to become coordinator
> 01:02:11,932 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: discovery took 0 ms, members: 1 rsps (0 coords) [done]
> 01:02:11,932 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,932 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> ...
> 01:02:11,941 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:02:11,941 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: nodes to choose new coord from are: [Test-NodeB-43694, Test-NodeA-29550]
> 01:02:11,941 TRACE (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: I (Test-NodeA-29550) am not the first of the nodes, waiting for another client to become coordinator
> 01:02:11,942 WARN (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: too many JOIN attempts (10): becoming singleton
> 01:02:11,942 DEBUG (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: installing view [Test-NodeA-29550|0] (1) [Test-NodeA-29550]
> 01:02:11,977 DEBUG (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-43694: created cluster (first member). My view is [Test-NodeB-43694|0], impl is org.jgroups.protocols.pbcast.CoordGmsImpl
> 01:02:11,977 DEBUG (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-29550: created cluster (first member). My view is [Test-NodeA-29550|0], impl is org.jgroups.protocols.pbcast.CoordGmsImpl
> {noformat}
> The problem seems to be that it takes longer for the coordinator to install the initial view and update {{LOCAL_PING}}'s {{PingData}} then it takes the other node to retry the discovery process 10 times.
> In some cases there is no retry, because one node starts slightly faster, but it's not yet coordinator when the 2nd node does its discovery, and both nodes decide they should be coordinator:
> {noformat}
> 01:13:44,460 INFO (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-5386: no members discovered after 3 ms: creating cluster as first member
> 01:13:44,463 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: discovery took 1 ms, members: 1 rsps (0 coords) [done]
> 01:13:44,465 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: could not determine coordinator from rsps 1 rsps (0 coords) [done]
> 01:13:44,465 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: nodes to choose new coord from are: [Test-NodeB-51165, Test-NodeA-5386]
> 01:13:44,466 TRACE (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: I (Test-NodeB-51165) am the first of the nodes, will become coordinator
> 01:13:44,466 DEBUG (ForkThread-2,ConcurrentStartTest:[]) [GMS] Test-NodeB-51165: installing view [Test-NodeB-51165|0] (1) [Test-NodeB-51165]
> 01:13:44,466 DEBUG (ForkThread-1,ConcurrentStartTest:[]) [GMS] Test-NodeA-5386: installing view [Test-NodeA-5386|0] (1) [Test-NodeA-5386]
> {noformat}
> This second failure mode seems to go away if I move the {{discovery}} map access inside the {{synchronized}} block both in {{findMembers()}} and in {{down()}}.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (JGRP-2398) LOCAL_PING should remove local PingData on stop
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2398?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2398 at 11/11/19 10:26 AM:
-----------------------------------------------------------
[~dan.berindei] {{LOCAL_PING.handleDisconnect()}} removes the local PingData. This method is called _before_ stop().
IMO, stop() is not required as handleDisconnect() does the same.
How do you reproduce this?
was (Author: belaban):
[~dan.berindei] {{LOCAL_PING.handleDisconnect()}} removes the local PingData. This method is called _before_ stop().
IMO, stop() is not required as handleDisconnect() does the same.
> LOCAL_PING should remove local PingData on stop
> -----------------------------------------------
>
> Key: JGRP-2398
> URL: https://issues.jboss.org/browse/JGRP-2398
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.1.6
> Reporter: Dan Berindei
> Assignee: Bela Ban
> Priority: Major
> Fix For: 4.1.8
>
>
> {{LOCAL_PING}} does not remove the local node's {{PING_DATA}} from the {{discovery}} map on stop.
> This means when a cluster name is reused, e.g. because we create a new cluster for each method in a test class, the nodes in the new cluster first try to connect to the dead coordinators of the old clusters. With the default {{GMS.max_join_attempts=10}}, this can take a really long time:
> {noformat}
> 13:53:39,024 DEBUG (testng-Test:[]) [GMS] address=Test-NodeA-57106, cluster=org.infinispan.partitionhandling.Test, physical address=127.0.0.1:55320
> 13:53:39,026 TRACE (testng-Test:[]) [GMS] Test-NodeA-57106: discovery took 0 ms, members: 17 rsps (4 coords) [done]
> 13:53:39,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: found multiple coords: [054b27e3-9308-65bf-de26-d6cdd4f6bdd3, 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca, Test-NodeA-37104, Test-NodeA-8303]
> 13:53:39,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca
> 13:53:41,026 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca timed out (after 2000 ms), on try 0
> 13:53:41,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to Test-NodeA-37104
> 13:53:43,026 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to Test-NodeA-37104 timed out (after 2000 ms), on try 0
> 13:53:43,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3
> 13:53:45,027 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3 timed out (after 2000 ms), on try 0
> 13:53:45,027 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to Test-NodeA-8303
> 13:53:47,027 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to Test-NodeA-8303 timed out (after 2000 ms), on try 0
> ...
> 13:54:51,036 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: found multiple coords: [054b27e3-9308-65bf-de26-d6cdd4f6bdd3, 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca, 0eb381de-3e9a-0320-b2d9-5db3cb39bb34, Test-NodeA-8303]
> 13:54:51,036 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3
> 13:54:53,036 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3 timed out (after 2000 ms), on try 9
> 13:54:53,036 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 0eb381de-3e9a-0320-b2d9-5db3cb39bb34
> 13:54:55,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 0eb381de-3e9a-0320-b2d9-5db3cb39bb34 timed out (after 2000 ms), on try 9
> 13:54:55,037 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca
> 13:54:57,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca timed out (after 2000 ms), on try 9
> 13:54:57,037 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to Test-NodeA-8303
> 13:54:59,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to Test-NodeA-8303 timed out (after 2000 ms), on try 9
> 13:54:59,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: too many JOIN attempts (10): becoming singleton
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (DROOLS-4731) Working with Long field
by Pavel Tavoda (Jira)
[ https://issues.jboss.org/browse/DROOLS-4731?page=com.atlassian.jira.plugi... ]
Pavel Tavoda commented on DROOLS-4731:
--------------------------------------
Yes, it makes no sense and it's provided only for clarification. But here are some strange cases which I expected to work:
1)
LongInRuleMvel.drl - Should work even in strict mode
LongError.gdst - Value should be translated to LONG constant (with l postfix) it's defined in XML as LONG only translator from GDST -> RULE ignores it
2) It's generally OK only I don't see place where to specify drools.dialect.mvel.strict=false. On maven command line it's not always right place and it's not working in maven config, maven plugin config and also not in kmodule.xml config. Documentation is silent about this.
3)
IntInRuleMvel.drl - should work (but OK it's 50/50 case) you can always postfix with l. However if you want be consistent than this scenario should also fail in 2) ;-)
LongError.gdst - same problem as 1)
> Working with Long field
> -----------------------
>
> Key: DROOLS-4731
> URL: https://issues.jboss.org/browse/DROOLS-4731
> Project: Drools
> Issue Type: Bug
> Components: core engine
> Affects Versions: 7.28.0.Final
> Reporter: Pavel Tavoda
> Assignee: Mario Fusco
> Priority: Major
> Attachments: error-with-long.zip
>
>
> Setting long field value in THEN:
> This are 3 different problems:
> 1) package as kjar with 'mvn clean install' result in "Error: unable to resolve method using strict-mode: org.drools.core.spi.Activation.setValue(java.lang.Integer)"
> RESULT:
> IntInRuleJava.drl - FAIL
> IntInRuleMvel.drl - FAIL
> LongInRuleJava.drl - OK
> LongInRuleMvel.drl - FAIL
> LongError.gdst - FAIL
> 2) package as kjar with 'mvn clean install -Ddrools.dialect.mvel.strict=false', same error on different files:
> IntInRuleJava.drl - FAIL
> IntInRuleMvel.drl - OK
> LongInRuleJava.drl - OK
> LongInRuleMvel.drl - OK
> LongError.gdst - OK
> 3) with 'mvn clean install -DgenerateModel=YES' it result in 'incompatible types: int cannot be converted to java.lang.Long', setting strict false have no influence on result
> IntInRuleJava.drl - FAIL
> IntInRuleMvel.drl - FAIL
> LongInRuleJava.drl - OK
> LongInRuleMvel.drl - OK
> LongError.gdst - FAIL
> Setting 'drools.dialect.mvel.strict' in kmodule.xml doesn't work.
> Example project attached.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months
[JBoss JIRA] (JGRP-2398) LOCAL_PING should remove local PingData on stop
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2398?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2398:
--------------------------------
[~dan.berindei] {{LOCAL_PING.handleDisconnect()}} removes the local PingData. This method is called _before_ stop().
IMO, stop() is not required as handleDisconnect() does the same.
> LOCAL_PING should remove local PingData on stop
> -----------------------------------------------
>
> Key: JGRP-2398
> URL: https://issues.jboss.org/browse/JGRP-2398
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.1.6
> Reporter: Dan Berindei
> Assignee: Bela Ban
> Priority: Major
> Fix For: 4.1.8
>
>
> {{LOCAL_PING}} does not remove the local node's {{PING_DATA}} from the {{discovery}} map on stop.
> This means when a cluster name is reused, e.g. because we create a new cluster for each method in a test class, the nodes in the new cluster first try to connect to the dead coordinators of the old clusters. With the default {{GMS.max_join_attempts=10}}, this can take a really long time:
> {noformat}
> 13:53:39,024 DEBUG (testng-Test:[]) [GMS] address=Test-NodeA-57106, cluster=org.infinispan.partitionhandling.Test, physical address=127.0.0.1:55320
> 13:53:39,026 TRACE (testng-Test:[]) [GMS] Test-NodeA-57106: discovery took 0 ms, members: 17 rsps (4 coords) [done]
> 13:53:39,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: found multiple coords: [054b27e3-9308-65bf-de26-d6cdd4f6bdd3, 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca, Test-NodeA-37104, Test-NodeA-8303]
> 13:53:39,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca
> 13:53:41,026 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca timed out (after 2000 ms), on try 0
> 13:53:41,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to Test-NodeA-37104
> 13:53:43,026 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to Test-NodeA-37104 timed out (after 2000 ms), on try 0
> 13:53:43,026 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3
> 13:53:45,027 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3 timed out (after 2000 ms), on try 0
> 13:53:45,027 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to Test-NodeA-8303
> 13:53:47,027 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to Test-NodeA-8303 timed out (after 2000 ms), on try 0
> ...
> 13:54:51,036 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: found multiple coords: [054b27e3-9308-65bf-de26-d6cdd4f6bdd3, 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca, 0eb381de-3e9a-0320-b2d9-5db3cb39bb34, Test-NodeA-8303]
> 13:54:51,036 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3
> 13:54:53,036 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 054b27e3-9308-65bf-de26-d6cdd4f6bdd3 timed out (after 2000 ms), on try 9
> 13:54:53,036 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 0eb381de-3e9a-0320-b2d9-5db3cb39bb34
> 13:54:55,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 0eb381de-3e9a-0320-b2d9-5db3cb39bb34 timed out (after 2000 ms), on try 9
> 13:54:55,037 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca
> 13:54:57,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to 6d7f9fb2-f56b-1b88-7f9d-cdee430ac8ca timed out (after 2000 ms), on try 9
> 13:54:57,037 DEBUG (testng-Test:[]) [GMS] Test-NodeA-57106: sending JOIN(Test-NodeA-57106) to Test-NodeA-8303
> 13:54:59,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: JOIN(Test-NodeA-57106) sent to Test-NodeA-8303 timed out (after 2000 ms), on try 9
> 13:54:59,037 WARN (testng-Test:[]) [GMS] Test-NodeA-57106: too many JOIN attempts (10): becoming singleton
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 8 months