<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body><div>I just got JGroups/Infinispan with JDBC_PING working from inside a Docker cluster in ECS on EC2. I use JDBC_PING rather than S3_PING, since I need a database anyway and didn't want to have to set up an S3 bucket just for this one purpose. Nicolás, if you're on AWS the default UDP transport for JGroups doesn't work because multicast isn't supported inside EC2, which may be your problem.<br></div>
<div> </div>
<div>Here are the configurations you'd need:<br></div>
<div> </div>
<div>1. The JGroups module has to reference to the db module. So in jgroups-module.xml I have:<br></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <dependencies><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <module name="javax.api"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <module name="org.postgresql.jdbc"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </dependencies></span><br></div>
<div> </div>
<div>2. The standalone-ha.xml has a JGroups subsystem (with TCP and JDBC_PING) that looks like the configuration below; I read certain variables from the environment, but may use the Wildfly vault tool for some of them. The external_addr property configurations are only needed if you're inside a Docker container, since Wildfly has to read the address of the EC2 instance hosting the container to register itself with JGroups. For the initialize_sql you can generally use the default, but for Postgres I needed a custom DDL because I needed the BYTEA data type which isn't in the default DDL.<br></div>
<div> </div>
<div> <span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"><subsystem xmlns="urn:jboss:domain:jgroups:4.0"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <channels default="ee"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <channel name="ee" stack="tcp"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </channels><br></span></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <stacks default="tcp"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <stack name="tcp"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <transport type="TCP" socket-binding="jgroups-tcp"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="external_addr">${env.EXTERNAL_HOST_IP}</property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </transport><br></span></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="JDBC_PING"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="connection_driver">org.postgresql.Driver</property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="connection_url">jdbc:postgresql://${env.POSTGRES_TCP_ADDR}:${env.POSTGRES_TCP_PORT}/${env.POSTGRES_DATABASE}</property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="connection_username">${env.POSTGRES_USER}</property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="connection_password">${env.POSTGRES_PASSWORD}</property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="initialize_sql"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> CREATE TABLE IF NOT EXISTS jgroupsping (<br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> own_addr VARCHAR(200) NOT NULL,<br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> cluster_name VARCHAR(200) NOT NULL,<br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> ping_data BYTEA DEFAULT NULL,<br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> PRIMARY KEY (own_addr, cluster_name)<br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> )<br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </protocol><br></span></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="MERGE3"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <property name="external_addr">${env.EXTERNAL_HOST_IP}</property><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </protocol><br></span></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="FD"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="VERIFY_SUSPECT"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="pbcast.NAKACK2"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="UNICAST3"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="pbcast.STABLE"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="pbcast.GMS"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="MFC"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <protocol type="FRAG2"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </stack><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </stacks><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> </subsystem></span><br></div>
<div> </div>
<div>3. If you're in a Docker container, you have to expose the JGroups ports so they are visible from outside the container, so in standalone-ha.xml in the socket bindings I have changed to the public interface:<br></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <socket-binding name="jgroups-tcp" interface="public" port="7600"/><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> <socket-binding name="jgroups-tcp-fd" interface="public" port="57600"/></span><br></div>
<div> </div>
<div>4. For Docker, the startup script needs to pass the EXTERNAL_HOST_IP variable. I have a wrapper start script that first queries the AWS instance metadata service for the host's private IP address:<br></div>
<div> </div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> export EXTERNAL_HOST_IP=$(curl -s 169.254.169.254/latest/meta-data/local-ipv4)</span><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"><br></span></div>
<div><span class="font" style="font-family: menlo, consolas, "courier new", monospace, sans-serif;"> exec $WILDFLY_HOME/bin/standalone.sh -c standalone-keycloak-ha.xml -Djboss.node.name=$HOSTNAME -Djgroups.bind_addr=global -b $HOSTNAME</span><br></div>
<div> </div>
<div>--------------------------------------------------------------------------------------------------------------------------------------------</div>
<div>From: <keycloak-user-bounces@lists.jboss.org> <br></div>
<div>Date: Wednesday, February 17, 2016 at 9:03 AM<br></div>
<div>To: "keycloak-user@lists.jboss.org" <keycloak-user@lists.jboss.org><br></div>
<div>Subject: [keycloak-user] Infinispan not working on HA environment with dockers.<br></div>
<div> </div>
<div>Hello all,<br></div>
<div>I'm trying to set a Keycloak HA environment up with dockers. I tried with jboss/keycloak-ha-postgres:1.8.0.Final image.<br></div>
<div> </div>
<div>I can't make infinispan work when I run 2 instances of my docker images. I get the following log in every node:<br></div>
<div> </div>
<div>Received new cluster view for channel ejb: [f9032dc82244|0] (1) [f9032dc82244]<br></div>
<div>Received new cluster view for channel hibernate: [f9032dc82244|0] (1) [f9032dc82244]<br></div>
<div>Received new cluster view for channel keycloak: [f9032dc82244|0] (1) [f9032dc82244]<br></div>
<div>Received new cluster view for channel web: [f9032dc82244|0] (1) [f9032dc82244]<br></div>
<div>Channel hibernate local address is f9032dc82244, physical addresses are [127.0.0.1:55200]<br></div>
<div>Channel keycloak local address is f9032dc82244, physical addresses are [127.0.0.1:55200]<br></div>
<div>Channel ejb local address is f9032dc82244, physical addresses are [127.0.0.1:55200]<br></div>
<div>Channel web local address is f9032dc82244, physical addresses are [127.0.0.1:55200]<br></div>
<div>Received new cluster view for channel server: [f9032dc82244|0] (1) [f9032dc82244]<br></div>
<div>Channel server local address is f9032dc82244, physical addresses are [127.0.0.1:55200]<br></div>
<div> </div>
<div>This is causing my user sessions are not shared between instances and it's not working properly.<br></div>
<div> </div>
<div>When I run 2 instances of keycloak without dockers, they work properly.<br></div>
<div> </div>
<div>Am I missing something? Is there any extra configuration that I need to change?<br></div>
<div> </div>
<div>Thanks,<br></div>
<div>Nicolas.-<br></div>
<pre>
--
http://www.fastmail.com - A fast, anti-spam email service.
</pre>
</body>
</html>