Have you disabled the UDP-related configurations? ____________________ Tonnis Wildeboer Autonomic.ai Engineering 650-204-0246 On 09/19/2017 07:19 AM, Jyoti Kumar Singh wrote: Hi, I have tried the JDBC_PING option which Tonnis has mentioned :- https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql After that I could see node discovery is happening but JOIN operation is getting timed out which eventually not forming clustering between two ECS instances. Is there any configuration am I missing here ? #Logs:- 2017-09-19 10:59:52,907 WARN [org.jgroups.protocols.UDP] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL JGRP000015: the receive buffer of socket ManagedMulticastSocketBinding was set to 25MB, but the OS only allocated 212.99KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux) 2017-09-19 10:59:59,475 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 1 2017-09-19 11:00:02,490 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 2 2017-09-19 11:00:05,508 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 3 2017-09-19 11:00:08,527 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 4 2017-09-19 11:00:11,542 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 5 2017-09-19 11:00:14,558 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 6 2017-09-19 11:00:17,579 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 7 2017-09-19 11:00:20,596 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 8 2017-09-19 11:00:23,611 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 9 2017-09-19 11:00:26,627 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to 16c566cfa08e timed out (after 3000 ms), on try 10 2017-09-19 11:00:26,627 WARN [org.jgroups.protocols.pbcast.GMS] (MSC service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL f0be09280f90: too many JOIN attempts (10): becoming singleton On Thu, Sep 14, 2017 at 10:48 PM, Jyoti Kumar Singh < jyoti.tech90(a)gmail.com&gt; wrote: > Hi Tonnis, > > Thank you very much for sharing the valuable information. I am checking > on this, hopefully I will also be able to achieve the HA. > > Thanks Again ! > > On Sep 14, 2017 10:00 PM, "Tonnis Wildeboer" <tonnis(a)autonomic.ai&gt; wrote: > >> Jyoti, >> >> I have been working on similar goal and was finally successful >> yesterday. We are using postgres and kubernetes. >> >> Here are the key sources of information that enabled me to succeed: >> >> The big key is here: >> https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql >> Use the .xsl templates here to transform on the standalone-ha.xml and >> you can see what is being done. >> >> I suggest that you simply use JDBC_PING, since you already have a shared >> database. >> I think it is instructive to understand what JDBC_PING (and JGroups in >> general) are doing: >> http://jgroups.org/manual4/index.html >> https://developer.jboss.org/wiki/JDBCPING >> >> You may benefit from this also, specifically, the need to bind >> jgroups-tcp and jgroups-tcp-fd to the proper interface. Not sure about your >> situation. >> >> --Tonnis >> >> ____________________ >> Tonnis Wildeboer >> Autonomic.ai Engineering >> >> On 09/14/2017 03:32 AM, Jyoti Kumar Singh wrote: >> >>> Hi Team, >>> >>> I am trying to enable high availability for Keycloak 3.1.0 on AWS ECS >>> instances. >>> >>> I am running two ECS instances in a cluster setup and also I have >>> setup Keycloak >>> in a clustered mode. To achieve this, I am using " >>> */standalone/configuration/standalone-ha.xml *" file while building the >>> docker image. Shared MySQL DB and Load Balancer setup are also in place. >>> >>> But when I checked Keycloak logs I am not seeing clustered nodes related >>> information in logs. I am seeing nodes are not able to see each other. >>> But >>> same settings are working fine in DCOS Marathon platform. >>> >>> Interestingly if I run two Keycloak instances in one AWS ECS instance on >>> different ports, I could see clustering related logs in Keycloak. >>> >>> Is there any standard guidelines which I can follow to achieve HA in AWS >>> ECS instance ?? I followed the below discussion thread but it didn't >>> help >>> me to fix the issue. >>> >>> #Link: >>> http://lists.jboss.org/pipermail/keycloak-user/2016-February >>> /004940.html >>> >>> >> -- *With Regards, Jyoti Kumar Singh*

Hi Tonnis, No, I have not disabled the UDP-related configurations. Here is the standalone-ha.xml which I am using as part of my configurations. On Tue, Sep 19, 2017 at 11:58 PM, Tonnis Wildeboer <tonnis(a)autonomic.ai <mailto:tonnis@autonomic.ai>> wrote: Have you disabled the UDP-related configurations? ____________________ Tonnis Wildeboer Autonomic.ai Engineering 650-204-0246 On 09/19/2017 07:19 AM, Jyoti Kumar Singh wrote: > Hi, > > I have tried the JDBC_PING option which Tonnis has mentioned :- > https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql > <https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql> > > After that I could see node discovery is happening but JOIN > operation is getting timed out which eventually not forming > clustering between two ECS instances. Is there any configuration > am I missing here ? > > > #Logs:- > > 2017-09-19 10:59:52,907 WARN [org.jgroups.protocols.UDP] (MSC > service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL > JGRP000015: the receive buffer of socket > ManagedMulticastSocketBinding was set to 25MB, but the OS only > allocated 212.99KB. This might lead to performance problems. > Please set your max receive buffer in the OS correctly (e.g. > net.core.rmem_max on Linux) > 2017-09-19 10:59:59,475 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 1 > 2017-09-19 11:00:02,490 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 2 > 2017-09-19 11:00:05,508 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 3 > 2017-09-19 11:00:08,527 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 4 > 2017-09-19 11:00:11,542 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 5 > 2017-09-19 11:00:14,558 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 6 > 2017-09-19 11:00:17,579 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 7 > 2017-09-19 11:00:20,596 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 8 > 2017-09-19 11:00:23,611 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 9 > 2017-09-19 11:00:26,627 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to > 16c566cfa08e timed out (after 3000 ms), on try 10 > 2017-09-19 11:00:26,627 WARN [org.jgroups.protocols.pbcast.GMS] > (MSC service thread 1-1) [f0be09280f90] KEYCLOAK > 3.1.0.Final-MySQL f0be09280f90: too many JOIN attempts (10): > becoming singleton > > On Thu, Sep 14, 2017 at 10:48 PM, Jyoti Kumar Singh > <jyoti.tech90(a)gmail.com <mailto:jyoti.tech90@gmail.com>> wrote: > > Hi Tonnis, > > Thank you very much for sharing the valuable information. I > am checking on this, hopefully I will also be able to achieve > the HA. > > Thanks Again ! > > On Sep 14, 2017 10:00 PM, "Tonnis Wildeboer" > <tonnis(a)autonomic.ai <mailto:tonnis@autonomic.ai>> wrote: > > Jyoti, > > I have been working on similar goal and was finally > successful yesterday. We are using postgres and kubernetes. > > Here are the key sources of information that enabled me > to succeed: > > The big key is here: > https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql > <https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql> > Use the .xsl templates here to transform on the > standalone-ha.xml and you can see what is being done. > > I suggest that you simply use JDBC_PING, since you > already have a shared database. > I think it is instructive to understand what JDBC_PING > (and JGroups in general) are doing: > http://jgroups.org/manual4/index.html > <http://jgroups.org/manual4/index.html> > https://developer.jboss.org/wiki/JDBCPING > <https://developer.jboss.org/wiki/JDBCPING> > > You may benefit from this also, specifically, the need to > bind jgroups-tcp and jgroups-tcp-fd to the proper > interface. Not sure about your situation. > > --Tonnis > > ____________________ > Tonnis Wildeboer > Autonomic.ai Engineering > > On 09/14/2017 03:32 AM, Jyoti Kumar Singh wrote: > > Hi Team, > > I am trying to enable high availability for Keycloak > 3.1.0 on AWS ECS > instances. > > I am running two ECS instances in a cluster setup and > also I have > setup Keycloak > in a clustered mode. To achieve this, I am using " > */standalone/configuration/standalone-ha.xml *" file > while building the > docker image. Shared MySQL DB and Load Balancer setup > are also in place. > > But when I checked Keycloak logs I am not seeing > clustered nodes related > information in logs. I am seeing nodes are not able > to see each other. But > same settings are working fine in DCOS Marathon platform. > > Interestingly if I run two Keycloak instances in one > AWS ECS instance on > different ports, I could see clustering related logs > in Keycloak. > > Is there any standard guidelines which I can follow > to achieve HA in AWS > ECS instance ?? I followed the below discussion > thread but it didn't help > me to fix the issue. > > #Link: > http://lists.jboss.org/pipermail/keycloak-user/2016-February/004940.html > <http://lists.jboss.org/pipermail/keycloak-user/2016-February/004940.html> > > > > > > -- > *With Regards, > Jyoti Kumar Singh* -- *With Regards, Jyoti Kumar Singh*