[keycloak-user] Using Keycloak in AWS EC2. What are people using? / Infinispan not working on HA environment withdockers.

[Aikeaguinea]

I just got JGroups/Infinispan with JDBC_PING working from inside a
Docker cluster in ECS on EC2. I use JDBC_PING rather than S3_PING, since
I need a database anyway and didn't want to have to set up an S3 bucket
just for this one purpose. Nicolás, if you're on AWS the default UDP
transport for JGroups doesn't work because multicast isn't supported
inside EC2, which may be your problem.

Here are the configurations you'd need:

1. The JGroups module has to reference to the db module. So in jgroups-
   module.xml I have:

<dependencies>        <module name="javax.api"/>       <module
name="org.postgresql.jdbc"/>    </dependencies>

2.The standalone-ha.xml has a JGroups subsystem (with TCP and JDBC_PING)
  that looks like the configuration below; I read certain variables from
  the environment, but may use the Wildfly vault tool for some of them.
  The external_addr property configurations are only needed if you're
  inside a Docker container, since Wildfly has to read the address of
  the EC2 instance hosting the container to register itself with
  JGroups. For the initialize_sql you can generally use the default, but
  for Postgres I needed a custom DDL because I needed the BYTEA data
  type which isn't in the default DDL.

<subsystem xmlns="urn:jboss:domain:jgroups:4.0">            <channels
default="ee">                <channel name="ee" stack="tcp"/>
</channels>

<stacks default="tcp">                <stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp">
<property name="external_addr">${env.EXTERNAL_HOST_IP}</property>
</transport>

<protocol type="JDBC_PING">                        <property
name="connection_driver">org.postgresql.Driver</property>
<property name="connection_url">jdbc:postgresql://${env.POSTGRES_TCP_AD-
DR}:${env.POSTGRES_TCP_PORT}/${env.POSTGRES_DATABASE}</property>
<property name="connection_username">${env.POSTGRES_USER}</property>
<property name="connection_password">${env.POSTGRES_PASSWORD}</property>
<property name="initialize_sql">                            CREATE TABLE
IF NOT EXISTS jgroupsping (                                own_addr
VARCHAR(200) NOT NULL,                                cluster_name
VARCHAR(200) NOT NULL,                                ping_data BYTEA
DEFAULT NULL,                                PRIMARY KEY (own_addr,
cluster_name)                            )
</property>                    </protocol>

<protocol type="MERGE3"/>                    <protocol type="FD_SOCK"
socket-binding="jgroups-tcp-fd">                        <property
name="external_addr">${env.EXTERNAL_HOST_IP}</property>
</protocol>

<protocol type="FD"/>                    <protocol
type="VERIFY_SUSPECT"/>                    <protocol
type="pbcast.NAKACK2"/>                    <protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>                    <protocol
type="pbcast.GMS"/>                    <protocol type="MFC"/>
<protocol type="FRAG2"/>                </stack>            </stacks>
</subsystem>

3. If you're in a Docker container, you have to expose the JGroups ports
   so they are visible from outside the container, so in standalone-
   ha.xml in the socket bindings I have changed to the public interface:

<socket-binding name="jgroups-tcp" interface="public" port="7600"/>
<socket-binding name="jgroups-tcp-fd" interface="public" port="57600"/>

4. For Docker, the startup script needs to pass the EXTERNAL_HOST_IP
   variable.  I have a wrapper start script that first queries the AWS
   instance metadata service for the host's private IP address:

export EXTERNAL_HOST_IP=$(curl -s 169.254.169.254/latest/meta-data/local-
ipv4)  exec $WILDFLY_HOME/bin/standalone.sh -c standalone-keycloak-
ha.xml -Djboss.node.name=$HOSTNAME -Djgroups.bind_addr=global -b
$HOSTNAME

------------------------------------------------------------------------
--------------------------------------------------------------------
From: <keycloak-user-bounces at lists.jboss.org> Date: Wednesday, February
17, 2016 at 9:03 AM To: "keycloak-user at lists.jboss.org" <keycloak-
user at lists.jboss.org> Subject: [keycloak-user] Infinispan not working on
HA environment with  dockers.

Hello all, I'm trying to set a Keycloak HA environment up with dockers.
I tried with jboss/keycloak-ha-postgres:1.8.0.Final image.

I can't make infinispan work when I run 2 instances of my docker images.
I get the following log in every node:

Received new cluster view for channel ejb: [f9032dc82244|0] (1)
[f9032dc82244] Received new cluster view for channel hibernate:
[f9032dc82244|0] (1) [f9032dc82244] Received new cluster view for
channel keycloak: [f9032dc82244|0] (1) [f9032dc82244] Received new
cluster view for channel web: [f9032dc82244|0] (1) [f9032dc82244]
Channel hibernate local address is f9032dc82244, physical addresses are
[127.0.0.1:55200] Channel keycloak local address is f9032dc82244,
physical addresses are [127.0.0.1:55200] Channel ejb local address is
f9032dc82244, physical addresses are [127.0.0.1:55200] Channel web local
address is f9032dc82244, physical addresses are [127.0.0.1:55200]
Received new cluster view for channel server: [f9032dc82244|0] (1)
[f9032dc82244] Channel server local address is f9032dc82244, physical
addresses are [127.0.0.1:55200]

This is causing my user sessions are not shared between instances and
it's not working properly.

When I run 2 instances of keycloak without dockers, they work properly.

Am I missing something? Is there any extra configuration that I need
to change?

Thanks, Nicolas.-

-- 
http://www.fastmail.com - A fast, anti-spam email service.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160217/16e07b30/attachment-0001.html