[keycloak-user] Enabling High Availability for Keycloak 3.1.0 on AWS ECS Instance
Tonnis Wildeboer
tonnis at autonomic.ai
Fri Sep 22 13:24:50 EDT 2017
Hi Jyoti,
Sorry I don't have time to help you debug this. But I would encourage
you to examine each piece of the ha-standalone.xml that is touched by
the xslt process, and also read up on the JDPC_PING and JGroups
documentation. Finally, be sure you have the set the JGROUPS_STACK
environment variable, or it will default to UDP, which is not fully
supported in AWS, and the whole reason we use JDBC_PING there rather
that the default.
See this in your xml file:
325 <subsystem xmlns="urn:jboss:domain:jgroups:4.0">
326 <channels default="ee">
327 <channel name="ee" stack="${env.JGROUPS_STACK:udp}"/>
328 </channels>
329 <stacks>
--Tonnis
On 09/20/2017 12:23 AM, Jyoti Kumar Singh wrote:
> Hi Tonnis,
>
> No, I have not disabled the UDP-related configurations. Here is the
> standalone-ha.xml which I am using as part of my configurations.
>
>
> On Tue, Sep 19, 2017 at 11:58 PM, Tonnis Wildeboer
> <tonnis at autonomic.ai <mailto:tonnis at autonomic.ai>> wrote:
>
> Have you disabled the UDP-related configurations?
>
> ____________________
> Tonnis Wildeboer
> Autonomic.ai Engineering
> 650-204-0246
>
> On 09/19/2017 07:19 AM, Jyoti Kumar Singh wrote:
>> Hi,
>>
>> I have tried the JDBC_PING option which Tonnis has mentioned :-
>> https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql
>> <https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql>
>>
>> After that I could see node discovery is happening but JOIN
>> operation is getting timed out which eventually not forming
>> clustering between two ECS instances. Is there any configuration
>> am I missing here ?
>>
>>
>> #Logs:-
>>
>> 2017-09-19 10:59:52,907 WARN [org.jgroups.protocols.UDP] (MSC
>> service thread 1-1) [f0be09280f90] KEYCLOAK 3.1.0.Final-MySQL
>> JGRP000015: the receive buffer of socket
>> ManagedMulticastSocketBinding was set to 25MB, but the OS only
>> allocated 212.99KB. This might lead to performance problems.
>> Please set your max receive buffer in the OS correctly (e.g.
>> net.core.rmem_max on Linux)
>> 2017-09-19 10:59:59,475 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 1
>> 2017-09-19 11:00:02,490 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 2
>> 2017-09-19 11:00:05,508 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 3
>> 2017-09-19 11:00:08,527 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 4
>> 2017-09-19 11:00:11,542 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 5
>> 2017-09-19 11:00:14,558 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 6
>> 2017-09-19 11:00:17,579 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 7
>> 2017-09-19 11:00:20,596 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 8
>> 2017-09-19 11:00:23,611 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 9
>> 2017-09-19 11:00:26,627 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: JOIN(f0be09280f90) sent to
>> 16c566cfa08e timed out (after 3000 ms), on try 10
>> 2017-09-19 11:00:26,627 WARN [org.jgroups.protocols.pbcast.GMS]
>> (MSC service thread 1-1) [f0be09280f90] KEYCLOAK
>> 3.1.0.Final-MySQL f0be09280f90: too many JOIN attempts (10):
>> becoming singleton
>>
>> On Thu, Sep 14, 2017 at 10:48 PM, Jyoti Kumar Singh
>> <jyoti.tech90 at gmail.com <mailto:jyoti.tech90 at gmail.com>> wrote:
>>
>> Hi Tonnis,
>>
>> Thank you very much for sharing the valuable information. I
>> am checking on this, hopefully I will also be able to achieve
>> the HA.
>>
>> Thanks Again !
>>
>> On Sep 14, 2017 10:00 PM, "Tonnis Wildeboer"
>> <tonnis at autonomic.ai <mailto:tonnis at autonomic.ai>> wrote:
>>
>> Jyoti,
>>
>> I have been working on similar goal and was finally
>> successful yesterday. We are using postgres and kubernetes.
>>
>> Here are the key sources of information that enabled me
>> to succeed:
>>
>> The big key is here:
>> https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql
>> <https://github.com/devsu/docker-keycloak/tree/master/server-ha-mysql>
>> Use the .xsl templates here to transform on the
>> standalone-ha.xml and you can see what is being done.
>>
>> I suggest that you simply use JDBC_PING, since you
>> already have a shared database.
>> I think it is instructive to understand what JDBC_PING
>> (and JGroups in general) are doing:
>> http://jgroups.org/manual4/index.html
>> <http://jgroups.org/manual4/index.html>
>> https://developer.jboss.org/wiki/JDBCPING
>> <https://developer.jboss.org/wiki/JDBCPING>
>>
>> You may benefit from this also, specifically, the need to
>> bind jgroups-tcp and jgroups-tcp-fd to the proper
>> interface. Not sure about your situation.
>>
>> --Tonnis
>>
>> ____________________
>> Tonnis Wildeboer
>> Autonomic.ai Engineering
>>
>> On 09/14/2017 03:32 AM, Jyoti Kumar Singh wrote:
>>
>> Hi Team,
>>
>> I am trying to enable high availability for Keycloak
>> 3.1.0 on AWS ECS
>> instances.
>>
>> I am running two ECS instances in a cluster setup and
>> also I have
>> setup Keycloak
>> in a clustered mode. To achieve this, I am using "
>> */standalone/configuration/standalone-ha.xml *" file
>> while building the
>> docker image. Shared MySQL DB and Load Balancer setup
>> are also in place.
>>
>> But when I checked Keycloak logs I am not seeing
>> clustered nodes related
>> information in logs. I am seeing nodes are not able
>> to see each other. But
>> same settings are working fine in DCOS Marathon platform.
>>
>> Interestingly if I run two Keycloak instances in one
>> AWS ECS instance on
>> different ports, I could see clustering related logs
>> in Keycloak.
>>
>> Is there any standard guidelines which I can follow
>> to achieve HA in AWS
>> ECS instance ?? I followed the below discussion
>> thread but it didn't help
>> me to fix the issue.
>>
>> #Link:
>> http://lists.jboss.org/pipermail/keycloak-user/2016-February/004940.html
>> <http://lists.jboss.org/pipermail/keycloak-user/2016-February/004940.html>
>>
>>
>>
>>
>>
>> --
>> *With Regards,
>> Jyoti Kumar Singh*
>
>
>
>
> --
> *With Regards,
> Jyoti Kumar Singh*
More information about the keycloak-user
mailing list