Re: [keycloak-user] Problem running keycloak cluster on EC2 with S3_ping

Friday, 18 December 2015

Well, i guess i must have done something wrong.

I tried launching my servers like you do :
/opt/keycloak-1.7.0.Final/bin/standalone.sh -c standalone-ha.xml
-Djboss.bind.address=10.1.7.103 -Djboss.bind.address.management=10.1.7.103
 -Djboss.socket.binding.port-offset=0 -Djboss.node.name=node0
and
 /opt/keycloak-1.7.0.Final/bin/standalone.sh -c standalone-ha.xml
-Djboss.bind.address=10.1.1.245 -Djboss.bind.address.management=10.1.1.245
 -Djboss.socket.binding.port-offset=0 -Djboss.node.name=node1

but i have the same problem, s3_ping creates 2 files and the nodes don't
see each other.

I just tried building the cluster using TCP_PING, this works fine.
When i get back to S3_ping it fails again.

Before using bind.address.management or bind.address, the IP addr was set
in the standalone-ha.xml in the last section of the file (this is hardcoded
for test purpose):

           <wsdl-host>${jboss.bind.address:127.0.0.1}</wsdl-host>
            <endpoint-config name="Standard-Endpoint-Config"/>
            <endpoint-config name="Recording-Endpoint-Config">
                <pre-handler-chain name="recording-handlers"
protocol-bindings="##SOAP11_HTTP ##SOAP11_HTTP_MTOM ##SOAP12_HTTP
##SOAP12_HTTP_MTOM">
                    <handler name="RecordingHandler"
class="org.jboss.ws.common.invocation.RecordingServerHandler"/>
                </pre-handler-chain>
            </endpoint-config>
            <client-config name="Standard-Client-Config"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:weld:2.0"/>
        <subsystem xmlns="urn:jboss:domain:keycloak-server:1.1">
            <web-context>auth</web-context>
        </subsystem>
    </profile>
    <interfaces>
        <interface name="management">
                <inet-address
value="${jboss.bind.address.management:10.1.1.245}"/>
        </interface>
        <interface name="public">
                <inet-address
value="${jboss.bind.address.management:10.1.1.245}"/>
        </interface>
        <!-- TODO - only show this if the jacorb subsystem is added  -->
        <interface name="unsecure">
            <!--
              ~  Used for IIOP sockets in the standard configuration.
              ~                  To secure JacORB you need to setup SSL
              -->
            <nic name="eth0"/>
        </interface>
    </interfaces>

Thanks again for trying, is it possible for you to send me you config file?
I'll just make a diff to see what i've done wrong.

CHARLES-EDOUARD GAGNAIRE
SysAdmin
c.gagnaire(a)kreactive.com
p. 06.27.80.28.53LYON "Le Capitole"
97, cours Gambetta
69481 Lyon Cedex 03

PARIS
16, rue de Turbigo
75002 Paris
[image: Kreactive] <http://www.kreactive.com/>

[image: Facebook] <https://www.facebook.com/kreactive> [image: Twitter]
<https://twitter.com/kreactive>

2015-12-18 16:55 GMT+01:00 Alan Field <afield(a)redhat.com&gt;:

...
 Hey Charles,

 I modified the standalone-ha.xml similar to your file to add the S3_PING
 protocol, and I was able to get two KeyCloak nodes to cluster. I did have
 to set the jboss.bind.address and jboss.bind.address.management. From your
 logs I can see that these addresses are both set in your environment as
 well. How are you setting these? My log is included below.

 Thanks,
 Alan

 [ec2-user@ip-172-31-4-165 keycloak-1.7.0.Final]$ bin/standalone.sh -c
 standalone-ha.xml -Djboss.socket.binding.port-offset=0 -Djboss.node.name=node0
 -Djboss.bind.address=$IP -Djboss.bind.address.management=$IP

 =========================================================================

 JBoss Bootstrap Environment

 JBOSS_HOME: /radargun/keycloak-1.7.0.Final

 JAVA: java

 JAVA_OPTS: -server -XX:+UseCompressedOops -server -XX:+UseCompressedOops
 -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true
 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

 =========================================================================

 Picked up _JAVA_OPTIONS:
 -Djgroups.s3.secret_access_key=ndWrqybZHIaFsEWEuMlWf5fJVffEapbu7DJgcYTd
 -Djgroups.s3.access_key=AKIAIVP76RKVWTPN6S6Q -Djgroups.s3.bucket=jdg-jgroups
 10:49:41,196 INFO [org.jboss.modules] (main) JBoss Modules version
 1.4.3.Final
 10:49:41,418 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
 10:49:41,497 INFO [org.jboss.as] (MSC service thread 1-7) WFLYSRV0049:
 Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) starting
 10:49:42,524 INFO [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 19) WFLYCTL0028: Attribute 'default-stack' in
 the resource at address '/subsystem=jgroups' is deprecated, and may be
 removed in future version. See the attribute description in the output of
 the read-resource-description operation to learn more about the deprecation.
 10:49:42,525 INFO [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 21) WFLYCTL0028: Attribute
 'job-repository-type' in the resource at address '/subsystem=batch' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 10:49:42,531 INFO [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 25) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=ExampleDS' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 10:49:42,534 INFO [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 25) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=KeycloakDS' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 10:49:42,707 INFO [org.jboss.as.server] (Controller Boot Thread)
 WFLYSRV0039: Creating http management service using socket-binding
 (management-http)
 10:49:42,741 INFO [org.xnio] (MSC service thread 1-2) XNIO version
 3.3.1.Final
 10:49:42,750 INFO [org.xnio.nio] (MSC service thread 1-2) XNIO NIO
 Implementation Version 3.3.1.Final
 10:49:42,782 INFO [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 39) WFLYCLINF0001: Activating Infinispan subsystem.
 10:49:42,782 INFO [org.wildfly.extension.io] (ServerService Thread Pool
 -- 38) WFLYIO001: Worker 'default' has auto-configured to 16 core threads
 with 128 task threads based on your 8 available processors
 10:49:42,784 INFO [org.jboss.remoting] (MSC service thread 1-2) JBoss
 Remoting version 4.0.9.Final
 10:49:42,800 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread
 Pool -- 43) WFLYCLJG0001: Activating JGroups subsystem.
 10:49:42,819 INFO [org.jboss.as.security] (ServerService Thread Pool --
 56) WFLYSEC0002: Activating Security Subsystem
 10:49:42,824 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 57)
 WFLYTX0013: Node identifier property is set to the default value. Please
 make sure it is unique.
 10:49:42,843 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 49)
 WFLYNAM0001: Activating Naming Subsystem
 10:49:42,841 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 46)
 WFLYJSF0007: Activated the following JSF Implementations: [main]
 10:49:42,859 INFO [org.jboss.as.security] (MSC service thread 1-3)
 WFLYSEC0001: Current PicketBox version=4.9.2.Final
 10:49:42,922 INFO [org.jboss.as.connector] (MSC service thread 1-8)
 WFLYJCA0009: Starting JCA Subsystem (IronJacamar 1.2.5.Final)
 10:49:42,976 INFO [org.jboss.as.webservices] (ServerService Thread Pool --
 59) WFLYWS0002: Activating WebServices Extension
 10:49:42,986 INFO [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 58) WFLYUT0003: Undertow 1.2.9.Final starting
 10:49:42,986 INFO [org.wildfly.extension.undertow] (MSC service thread
 1-5) WFLYUT0003: Undertow 1.2.9.Final starting
 10:49:42,991 INFO [org.jboss.as.connector.subsystems.datasources]
 (ServerService Thread Pool -- 35) WFLYJCA0004: Deploying JDBC-compliant
 driver class org.h2.Driver (version 1.3)
 10:49:42,994 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service
 thread 1-2) WFLYJCA0018: Started Driver service with driver-name = h2
 10:49:43,118 INFO [org.jboss.as.naming] (MSC service thread 1-7)
 WFLYNAM0003: Starting Naming Service
 10:49:43,119 INFO [org.jboss.as.mail.extension] (MSC service thread 1-6)
 WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
 10:49:43,168 INFO [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 58) WFLYUT0014: Creating file handler for path
 /radargun/keycloak-1.7.0.Final/welcome-content
 10:49:43,181 INFO [org.wildfly.extension.undertow] (MSC service thread
 1-7) WFLYUT0012: Started server default-server.
 10:49:43,213 INFO [org.wildfly.extension.undertow] (MSC service thread
 1-7) WFLYUT0018: Host default-host starting
 10:49:43,294 INFO [org.wildfly.extension.undertow] (MSC service thread
 1-8) WFLYUT0006: Undertow AJP listener ajp listening on /172.31.4.165:8009
 10:49:43,303 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 62)
 MODCLUSTER000001: Initializing mod_cluster version 1.3.1.Final
 10:49:43,307 INFO [org.wildfly.extension.undertow] (MSC service thread
 1-2) WFLYUT0006: Undertow HTTP listener default listening on /
 172.31.4.165:8080
 10:49:43,333 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 62)
 MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
 10:49:43,480 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-3) WFLYJCA0001: Bound data source
 [java:jboss/datasources/ExampleDS]
 10:49:43,480 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-4) WFLYJCA0001: Bound data source
 [java:jboss/datasources/KeycloakDS]
 10:49:43,739 INFO [org.jboss.as.server.deployment] (MSC service thread
 1-3) WFLYSRV0027: Starting deployment of "keycloak-server.war"
 (runtime-name: "keycloak-server.war")
 10:49:43,784 INFO [org.jboss.ws.common.management] (MSC service thread
 1-8) JBWS022052: Starting JBoss Web Services - Stack CXF Server 5.0.0.Final
 10:49:44,786 INFO [stdout] (MSC service thread 1-1)
 10:49:44,786 INFO [stdout] (MSC service thread 1-1)
 -------------------------------------------------------------------
 10:49:44,786 INFO [stdout] (MSC service thread 1-1) GMS: address=node0,
 cluster=ee, physical address=172.31.4.165:7600
 10:49:44,786 INFO [stdout] (MSC service thread 1-1)
 -------------------------------------------------------------------
 10:49:45,049 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
 Thread Pool -- 64) ISPN000078: Starting JGroups channel keycloak
 *10:49:45,055 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
 Thread Pool -- 64) ISPN000094: Received new cluster view for channel
 keycloak: [node0|0] (1) [node0]*
 10:49:45,064 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
 Thread Pool -- 64) ISPN000079: Channel keycloak local address is node0,
 physical addresses are [172.31.4.165:7600]
 10:49:45,067 INFO [org.infinispan.factories.GlobalComponentRegistry]
 (ServerService Thread Pool -- 64) ISPN000128: Infinispan version:
 Infinispan 'Insanely Bad Elf' 7.2.3.Final
 10:49:45,242 INFO [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 64) WFLYCLINF0002: Started users cache from keycloak
 container
 10:49:45,242 INFO [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 65) WFLYCLINF0002: Started realms cache from keycloak
 container
 10:49:45,243 INFO [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 66) WFLYCLINF0002: Started sessions cache from keycloak
 container
 10:49:45,243 INFO [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 62) WFLYCLINF0002: Started loginFailures cache from keycloak
 container
 10:49:45,846 INFO [org.keycloak.services.resources.KeycloakApplication]
 (ServerService Thread Pool -- 63) Load config from
 /radargun/keycloak-1.7.0.Final/standalone/configuration/keycloak-server.json
 10:49:46,642 INFO [org.hibernate.jpa.internal.util.LogHelper]
 (ServerService Thread Pool -- 63) HHH000204: Processing PersistenceUnitInfo
 [
 name: keycloak-default
 ...]
 10:49:46,695 INFO [org.hibernate.Version] (ServerService Thread Pool --
 63) HHH000412: Hibernate Core {4.3.10.Final}
 10:49:46,696 INFO [org.hibernate.cfg.Environment] (ServerService Thread
 Pool -- 63) HHH000206: hibernate.properties not found
 10:49:46,697 INFO [org.hibernate.cfg.Environment] (ServerService Thread
 Pool -- 63) HHH000021: Bytecode provider name : javassist
 10:49:46,856 INFO [org.hibernate.annotations.common.Version]
 (ServerService Thread Pool -- 63) HCANN000001: Hibernate Commons
 Annotations {4.0.5.Final}
 10:49:46,906 INFO [org.hibernate.dialect.Dialect] (ServerService Thread
 Pool -- 63) HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
 10:49:46,912 WARN [org.hibernate.dialect.H2Dialect] (ServerService Thread
 Pool -- 63) HHH000431: Unable to determine H2 database version, certain
 features may not work
 10:49:47,191 INFO
 [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ServerService
 Thread Pool -- 63) HHH000397: Using ASTQueryTranslatorFactory
 10:49:47,223 INFO [org.hibernate.validator.internal.util.Version]
 (ServerService Thread Pool -- 63) HV000001: Hibernate Validator 5.1.3.Final
 10:49:49,112 INFO [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 63) WFLYCLINF0002: Started offlineSessions cache from
 keycloak container
 10:49:49,147 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Deploying javax.ws.rs.core.Application:
 class org.keycloak.services.resources.KeycloakApplication
 10:49:49,149 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding class resource
 org.keycloak.services.resources.ThemeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,149 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding class resource
 org.keycloak.services.resources.JsResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,150 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding class resource
 org.keycloak.services.resources.WelcomeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,150 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding class resource
 org.keycloak.services.resources.QRCodeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,150 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding singleton resource
 org.keycloak.services.resources.ServerVersionResource from Application
 class org.keycloak.services.resources.KeycloakApplication
 10:49:49,151 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding provider singleton
 org.keycloak.services.util.ObjectMapperResolver from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,151 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding provider singleton
 org.keycloak.services.resources.ModelExceptionMapper from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,151 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding singleton resource
 org.keycloak.services.resources.admin.AdminRoot from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,152 INFO [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 63) Adding singleton resource
 org.keycloak.services.resources.RealmsResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 10:49:49,229 INFO [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 63) WFLYUT0021: Registered web context: /auth
 10:49:49,277 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61)
 WFLYSRV0010: Deployed "keycloak-server.war" (runtime-name :
 "keycloak-server.war")
 10:49:49,572 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060:
 Http management interface listening on http://172.31.4.165:9990/management
 10:49:49,573 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051:
 Admin console listening on http://172.31.4.165:9990
 10:49:49,573 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025:
 Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) started in 8688ms - Started
 344 of 593 services (339 services are lazy, passive or on-demand)
 *10:50:16,178 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (Incoming-2,ee,node0) ISPN000094: Received new cluster view for channel
 keycloak: [node0|1] (2) [node0, node1]*
 10:50:16,650 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t2)
 ISPN000310: Starting cluster-wide rebalance for cache users, topology
 CacheTopology{id=1, rebalanceId=1, currentCH=ReplicatedConsistentHash{ns =
 60, owners = (1)[node0: 60]}, pendingCH=ReplicatedConsistentHash{ns = 60,
 owners = (2)[node0: 30, node1: 30]}, unionCH=null, actualMembers=[node0,
 node1]}
 10:50:16,650 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t4)
 ISPN000310: Starting cluster-wide rebalance for cache realms, topology
 CacheTopology{id=1, rebalanceId=1, currentCH=ReplicatedConsistentHash{ns =
 60, owners = (1)[node0: 60]}, pendingCH=ReplicatedConsistentHash{ns = 60,
 owners = (2)[node0: 30, node1: 30]}, unionCH=null, actualMembers=[node0,
 node1]}
 10:50:16,652 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t1)
 ISPN000310: Starting cluster-wide rebalance for cache sessions, topology
 CacheTopology{id=1, rebalanceId=1, currentCH=DefaultConsistentHash{ns=80,
 owners = (1)[node0: 80+0]}, pendingCH=DefaultConsistentHash{ns=80, owners =
 (2)[node0: 40+0, node1: 40+0]}, unionCH=null, actualMembers=[node0, node1]}
 10:50:16,652 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3)
 ISPN000310: Starting cluster-wide rebalance for cache loginFailures,
 topology CacheTopology{id=1, rebalanceId=1,
 currentCH=DefaultConsistentHash{ns=80, owners = (1)[node0: 80+0]},
 pendingCH=DefaultConsistentHash{ns=80, owners = (2)[node0: 40+0, node1:
 40+0]}, unionCH=null, actualMembers=[node0, node1]}
 10:50:16,712 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t4)
 ISPN000336: Finished cluster-wide rebalance for cache users, topology id = 1
 10:50:16,712 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t1)
 ISPN000336: Finished cluster-wide rebalance for cache realms, topology id =
 1
 10:50:16,758 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t1)
 ISPN000336: Finished cluster-wide rebalance for cache sessions, topology id
 = 1
 10:50:16,761 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t4)
 ISPN000336: Finished cluster-wide rebalance for cache loginFailures,
 topology id = 1
 10:50:22,061 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t4)
 ISPN000310: Starting cluster-wide rebalance for cache offlineSessions,
 topology CacheTopology{id=1, rebalanceId=1,
 currentCH=DefaultConsistentHash{ns=80, owners = (1)[node0: 80+0]},
 pendingCH=DefaultConsistentHash{ns=80, owners = (2)[node0: 40+0, node1:
 40+0]}, unionCH=null, actualMembers=[node0, node1]}
 10:50:22,085 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t4)
 ISPN000336: Finished cluster-wide rebalance for cache offlineSessions,
 topology id = 1

 ------------------------------

 *From: *"Alan Field" <afield(a)redhat.com&gt;
 *To: *"charles-edouard gagnaire" <c.gagnaire(a)kreactive.com&gt;
 *Cc: *keycloak-user(a)lists.jboss.org
 *Sent: *Friday, December 18, 2015 10:21:54 AM

 *Subject: *Re: [keycloak-user] Problem running keycloak cluster on EC2
 with S3_ping

 Hey Charles,

 Thanks for the logs. I'm not sure what is wrong, but it looks like each
 server is creating a cluster of 1. I'll try it with my AWS account to see
 if I can figure out what is wrong.

 Alan

 ------------------------------

 *From: *"charles-edouard gagnaire" <c.gagnaire(a)kreactive.com&gt;
 *To: *"Alan Field" <afield(a)redhat.com&gt;
 *Cc: *keycloak-user(a)lists.jboss.org
 *Sent: *Friday, December 18, 2015 10:04:53 AM
 *Subject: *Re: [keycloak-user] Problem running keycloak cluster on EC2
 with S3_ping

 First i want to thank you guys for the quick answers, i was still reading
 the "Replace use of Infinispan with User Sessions SPI ?" discussion.

 Yes of course i can send all the logs. You'll find them below.

 The JGroups version is the one shipping with keycloak 1.7, but the problem
 was the same with Keycloak 1.6.
 Looking at the config file, it looks like i'm using : <subsystem
 xmlns="urn:jboss:domain:jgroups:3.0">

 I didn't mention it but i use the archive i found on Keycloak website. The
 archive is "keycloak-1.7.0.Final.tar.gz".
 I just untar and modify the config files, then i launched it using :
 /opt/keycloak-1.7.0.Final/bin/standalone.sh -c standalone-ha.xml

 Thank you again for your help

 The logs for server 1 are :
 =========================================================================

   JBoss Bootstrap Environment

   JBOSS_HOME: /opt/keycloak-1.7.0.Final

   JAVA: /usr/lib/jvm/jre/bin/java

   JAVA_OPTS:  -server -XX:+UseCompressedOops  -server
 -XX:+UseCompressedOops -Xms64m -Xmx512m -XX:MaxPermSize=256m
 -Djava.net.preferIPv4Stack=true
 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

 =========================================================================

 OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
 support was removed in 8.0
 14:38:44,910 INFO  [org.jboss.modules] (main) JBoss Modules version
 1.4.3.Final
 14:38:45,091 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
 14:38:45,163 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049:
 Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) starting
 14:38:46,358 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 24) WFLYCTL0028: Attribute
 'job-repository-type' in the resource at address '/subsystem=batch' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:46,360 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=PgDskeycloak' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:46,362 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=ExampleDS' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:46,362 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=KeycloakDS' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:46,370 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 21) WFLYCTL0028: Attribute 'default-stack' in
 the resource at address '/subsystem=jgroups' is deprecated, and may be
 removed in future version. See the attribute description in the output of
 the read-resource-description operation to learn more about the deprecation.
 14:38:46,572 INFO  [org.jboss.as.server] (Controller Boot Thread)
 WFLYSRV0039: Creating http management service using socket-binding
 (management-http)
 14:38:46,589 INFO  [org.xnio] (MSC service thread 1-4) XNIO version
 3.3.1.Final
 14:38:46,607 INFO  [org.xnio.nio] (MSC service thread 1-4) XNIO NIO
 Implementation Version 3.3.1.Final
 14:38:46,655 INFO  [org.jboss.remoting] (MSC service thread 1-4) JBoss
 Remoting version 4.0.9.Final
 14:38:46,687 INFO  [org.wildfly.extension.io] (ServerService Thread Pool
 -- 38) WFLYIO001: Worker 'default' has auto-configured to 4 core threads
 with 32 task threads based on your 2 available processors
 14:38:46,685 INFO  [org.jboss.as.connector.subsystems.datasources]
 (ServerService Thread Pool -- 35) WFLYJCA0005: Deploying non-JDBC-compliant
 driver class org.postgresql.Driver (version 9.4)
 14:38:46,715 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread
 Pool -- 43) WFLYCLJG0001: Activating JGroups subsystem.
 14:38:46,724 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 39) WFLYCLINF0001: Activating Infinispan subsystem.
 14:38:46,744 INFO  [org.jboss.as.connector] (MSC service thread 1-4)
 WFLYJCA0009: Starting JCA Subsystem (IronJacamar 1.2.5.Final)
 14:38:46,746 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service
 thread 1-2) WFLYJCA0018: Started Driver service with driver-name =
 postgresql
 14:38:46,767 INFO  [org.jboss.as.connector.subsystems.datasources]
 (ServerService Thread Pool -- 35) WFLYJCA0004: Deploying JDBC-compliant
 driver class org.h2.Driver (version 1.3)
 14:38:46,769 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service
 thread 1-2) WFLYJCA0018: Started Driver service with driver-name = h2
 14:38:46,781 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 46)
 WFLYJSF0007: Activated the following JSF Implementations: [main]
 14:38:46,772 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 49)
 WFLYNAM0001: Activating Naming Subsystem
 14:38:46,914 INFO  [org.jboss.as.security] (ServerService Thread Pool --
 56) WFLYSEC0002: Activating Security Subsystem
 14:38:46,916 INFO  [org.jboss.as.security] (MSC service thread 1-1)
 WFLYSEC0001: Current PicketBox version=4.9.2.Final
 14:38:46,932 WARN  [org.jboss.as.txn] (ServerService Thread Pool -- 57)
 WFLYTX0013: Node identifier property is set to the default value. Please
 make sure it is unique.
 14:38:46,957 INFO  [org.jboss.as.webservices] (ServerService Thread Pool
 -- 59) WFLYWS0002: Activating WebServices Extension
 14:38:46,985 INFO  [org.jboss.as.naming] (MSC service thread 1-2)
 WFLYNAM0003: Starting Naming Service
 14:38:46,992 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-4)
 WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
 14:38:47,115 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-1) WFLYUT0003: Undertow 1.2.9.Final starting
 14:38:47,119 INFO  [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 58) WFLYUT0003: Undertow 1.2.9.Final starting
 14:38:47,206 INFO  [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 58) WFLYUT0014: Creating file handler for path
 /opt/keycloak-1.7.0.Final/welcome-content
 14:38:47,229 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-4) WFLYUT0012: Started server default-server.
 14:38:47,263 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-4) WFLYUT0018: Host default-host starting
 14:38:47,320 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-2) WFLYUT0006: Undertow AJP listener ajp listening on /10.1.7.103:8009
 14:38:47,324 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-3) WFLYUT0006: Undertow HTTP listener default listening on /
 10.1.7.103:8080
 14:38:47,339 INFO  [org.jboss.modcluster] (ServerService Thread Pool --
 62) MODCLUSTER000001: Initializing mod_cluster version 1.3.1.Final
 14:38:47,372 INFO  [org.jboss.modcluster] (ServerService Thread Pool --
 62) MODCLUSTER000032: Listening to proxy advertisements on /
 224.0.1.105:23364
 14:38:47,478 INFO

[org.jboss.as.connector.subsystems.datasources.AbstractDataSourceService$AS7DataSourceDeployer]
 (MSC service thread 1-1) IJ020018: Enabling <validate-on-match> for
 java:jboss/datasources/PgDskeycloak
 14:38:47,513 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-2) WFLYJCA0001: Bound data source
 [java:jboss/datasources/KeycloakDS]
 14:38:47,513 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-2) WFLYJCA0001: Bound data source
 [java:jboss/datasources/ExampleDS]
 14:38:47,530 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-2) WFLYJCA0001: Bound data source
 [java:jboss/datasources/PgDskeycloak]
 14:38:47,673 INFO  [org.jboss.as.server.deployment] (MSC service thread
 1-4) WFLYSRV0027: Starting deployment of "keycloak-server.war"
 (runtime-name: "keycloak-server.war")
 14:38:47,820 INFO  [org.jboss.ws.common.management] (MSC service thread
 1-3) JBWS022052: Starting JBoss Web Services - Stack CXF Server 5.0.0.Final
 14:38:48,898 INFO  [stdout] (MSC service thread 1-2)
 14:38:48,898 INFO  [stdout] (MSC service thread 1-2)
 -------------------------------------------------------------------
 14:38:48,898 INFO  [stdout] (MSC service thread 1-2) GMS:
 address=ip-10-1-7-103, cluster=ee, physical address=10.1.7.103:7600
 14:38:48,899 INFO  [stdout] (MSC service thread 1-2)
 -------------------------------------------------------------------
 14:38:49,250 INFO
  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (ServerService Thread Pool -- 62) ISPN000078: Starting JGroups channel
 keycloak
 14:38:49,265 INFO
  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (ServerService Thread Pool -- 62) ISPN000094: Received new cluster view for
 channel keycloak: [ip-10-1-7-103|0] (1) [ip-10-1-7-103]
 14:38:49,273 INFO
  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (ServerService Thread Pool -- 62) ISPN000079: Channel keycloak local
 address is ip-10-1-7-103, physical addresses are [10.1.7.103:7600]
 14:38:49,277 INFO  [org.infinispan.factories.GlobalComponentRegistry]
 (ServerService Thread Pool -- 62) ISPN000128: Infinispan version:
 Infinispan 'Insanely Bad Elf' 7.2.3.Final
 14:38:49,521 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 65) WFLYCLINF0002: Started users cache from keycloak
 container
 14:38:49,529 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 62) WFLYCLINF0002: Started loginFailures cache from keycloak
 container
 14:38:49,530 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 66) WFLYCLINF0002: Started sessions cache from keycloak
 container
 14:38:49,536 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 63) WFLYCLINF0002: Started realms cache from keycloak
 container
 14:38:50,116 INFO  [org.keycloak.services.resources.KeycloakApplication]
 (ServerService Thread Pool -- 66) Load config from
 /opt/keycloak-1.7.0.Final/standalone/configuration/keycloak-server.json
 14:38:50,638 INFO  [org.hibernate.jpa.internal.util.LogHelper]
 (ServerService Thread Pool -- 66) HHH000204: Processing PersistenceUnitInfo
 [
         name: keycloak-default
         ...]
 14:38:50,690 INFO  [org.hibernate.Version] (ServerService Thread Pool --
 66) HHH000412: Hibernate Core {4.3.10.Final}
 14:38:50,691 INFO  [org.hibernate.cfg.Environment] (ServerService Thread
 Pool -- 66) HHH000206: hibernate.properties not found
 14:38:50,693 INFO  [org.hibernate.cfg.Environment] (ServerService Thread
 Pool -- 66) HHH000021: Bytecode provider name : javassist
 14:38:50,842 INFO  [org.hibernate.annotations.common.Version]
 (ServerService Thread Pool -- 66) HCANN000001: Hibernate Commons
 Annotations {4.0.5.Final}
 14:38:51,794 INFO  [org.hibernate.dialect.Dialect] (ServerService Thread
 Pool -- 66) HHH000400: Using dialect:
 org.hibernate.dialect.PostgreSQL9Dialect
 14:38:51,803 INFO  [org.hibernate.engine.jdbc.internal.LobCreatorBuilder]
 (ServerService Thread Pool -- 66) HHH000424: Disabling contextual LOB
 creation as createClob() method threw error :
 java.lang.reflect.InvocationTargetException
 14:38:52,120 INFO
  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ServerService
 Thread Pool -- 66) HHH000397: Using ASTQueryTranslatorFactory
 14:38:52,156 INFO  [org.hibernate.validator.internal.util.Version]
 (ServerService Thread Pool -- 66) HV000001: Hibernate Validator 5.1.3.Final
 14:38:53,706 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 66) WFLYCLINF0002: Started offlineSessions cache from
 keycloak container
 14:38:53,748 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Deploying javax.ws.rs.core.Application:
 class org.keycloak.services.resources.KeycloakApplication
 14:38:53,750 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding class resource
 org.keycloak.services.resources.WelcomeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,750 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding class resource
 org.keycloak.services.resources.JsResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,750 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding class resource
 org.keycloak.services.resources.QRCodeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,750 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding class resource
 org.keycloak.services.resources.ThemeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,751 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding singleton resource
 org.keycloak.services.resources.RealmsResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,751 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding singleton resource
 org.keycloak.services.resources.ServerVersionResource from Application
 class org.keycloak.services.resources.KeycloakApplication
 14:38:53,751 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding singleton resource
 org.keycloak.services.resources.admin.AdminRoot from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,751 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding provider singleton
 org.keycloak.services.util.ObjectMapperResolver from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,752 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 66) Adding provider singleton
 org.keycloak.services.resources.ModelExceptionMapper from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:38:53,824 INFO  [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 66) WFLYUT0021: Registered web context: /auth
 14:38:53,920 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 61)
 WFLYSRV0010: Deployed "keycloak-server.war" (runtime-name :
 "keycloak-server.war")
 14:38:54,021 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060:
 Http management interface listening on http://10.1.7.103:9990/management
 14:38:54,021 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051:
 Admin console listening on http://10.1.7.103:9990
 14:38:54,022 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025:
 Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) started in 9388ms - Started
 349 of 613 services (353 services are lazy, passive or on-demand)

 The logs for server 2 are :
 =========================================================================

   JBoss Bootstrap Environment

   JBOSS_HOME: /opt/keycloak-1.7.0.Final

   JAVA: /usr/lib/jvm/jre/bin/java

   JAVA_OPTS:  -server -XX:+UseCompressedOops  -server
 -XX:+UseCompressedOops -Xms64m -Xmx512m -XX:MaxPermSize=256m
 -Djava.net.preferIPv4Stack=true
 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

 =========================================================================

 14:38:48,239 INFO  [org.jboss.modules] (main) JBoss Modules version
 1.4.3.Final
 14:38:48,723 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
 14:38:48,896 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049:
 Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) starting
 14:38:50,979 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 8) WFLYCTL0028: Attribute
 'job-repository-type' in the resource at address '/subsystem=batch' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:50,983 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 11) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=PgDskeycloak' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:50,986 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 11) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=ExampleDS' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:51,010 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'default-stack' in
 the resource at address '/subsystem=jgroups' is deprecated, and may be
 removed in future version. See the attribute description in the output of
 the read-resource-description operation to learn more about the deprecation.
 14:38:51,044 INFO  [org.jboss.as.controller.management-deprecated]
 (ServerService Thread Pool -- 11) WFLYCTL0028: Attribute 'enabled' in the
 resource at address '/subsystem=datasources/data-source=KeycloakDS' is
 deprecated, and may be removed in future version. See the attribute
 description in the output of the read-resource-description operation to
 learn more about the deprecation.
 14:38:51,452 INFO  [org.jboss.as.server] (Controller Boot Thread)
 WFLYSRV0039: Creating http management service using socket-binding
 (management-http)
 14:38:51,499 INFO  [org.xnio] (MSC service thread 1-1) XNIO version
 3.3.1.Final
 14:38:51,520 INFO  [org.xnio.nio] (MSC service thread 1-1) XNIO NIO
 Implementation Version 3.3.1.Final
 14:38:51,590 INFO  [org.jboss.as.connector.subsystems.datasources]
 (ServerService Thread Pool -- 35) WFLYJCA0005: Deploying non-JDBC-compliant
 driver class org.postgresql.Driver (version 9.4)
 14:38:51,603 INFO  [org.wildfly.extension.io] (ServerService Thread Pool
 -- 38) WFLYIO001: Worker 'default' has auto-configured to 2 core threads
 with 16 task threads based on your 1 available processors
 14:38:51,601 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 39) WFLYCLINF0001: Activating Infinispan subsystem.
 14:38:51,634 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread
 Pool -- 43) WFLYCLJG0001: Activating JGroups subsystem.
 14:38:51,694 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 49)
 WFLYNAM0001: Activating Naming Subsystem
 14:38:51,666 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 46)
 WFLYJSF0007: Activated the following JSF Implementations: [main]
 14:38:51,696 INFO  [org.jboss.as.connector] (MSC service thread 1-2)
 WFLYJCA0009: Starting JCA Subsystem (IronJacamar 1.2.5.Final)
 14:38:51,932 INFO  [org.jboss.as.webservices] (ServerService Thread Pool
 -- 59) WFLYWS0002: Activating WebServices Extension
 14:38:51,970 INFO  [org.jboss.remoting] (MSC service thread 1-1) JBoss
 Remoting version 4.0.9.Final
 14:38:51,975 INFO  [org.jboss.as.security] (ServerService Thread Pool --
 56) WFLYSEC0002: Activating Security Subsystem
 14:38:51,972 INFO  [org.jboss.as.connector.subsystems.datasources]
 (ServerService Thread Pool -- 35) WFLYJCA0004: Deploying JDBC-compliant
 driver class org.h2.Driver (version 1.3)
 14:38:51,971 WARN  [org.jboss.as.txn] (ServerService Thread Pool -- 57)
 WFLYTX0013: Node identifier property is set to the default value. Please
 make sure it is unique.
 14:38:52,140 INFO  [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 58) WFLYUT0003: Undertow 1.2.9.Final starting
 14:38:52,187 INFO  [org.jboss.as.security] (MSC service thread 1-2)
 WFLYSEC0001: Current PicketBox version=4.9.2.Final
 14:38:52,224 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service
 thread 1-1) WFLYJCA0018: Started Driver service with driver-name =
 postgresql
 14:38:52,225 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service
 thread 1-1) WFLYJCA0018: Started Driver service with driver-name = h2
 14:38:52,368 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-1) WFLYUT0003: Undertow 1.2.9.Final starting
 14:38:52,369 INFO  [org.jboss.as.naming] (MSC service thread 1-2)
 WFLYNAM0003: Starting Naming Service
 14:38:52,471 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-2)
 WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
 14:38:52,710 INFO  [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 58) WFLYUT0014: Creating file handler for path
 /opt/keycloak-1.7.0.Final/welcome-content
 14:38:52,864 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-2) WFLYUT0012: Started server default-server.
 14:38:53,133 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-2) WFLYUT0006: Undertow HTTP listener default listening on /
 10.1.1.245:8080
 14:38:53,166 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-2) WFLYUT0018: Host default-host starting
 14:38:53,192 INFO  [org.wildfly.extension.undertow] (MSC service thread
 1-2) WFLYUT0006: Undertow AJP listener ajp listening on /10.1.1.245:8009
 14:38:53,211 INFO  [org.jboss.modcluster] (ServerService Thread Pool --
 62) MODCLUSTER000001: Initializing mod_cluster version 1.3.1.Final
 14:38:53,307 INFO  [org.jboss.modcluster] (ServerService Thread Pool --
 62) MODCLUSTER000032: Listening to proxy advertisements on /
 224.0.1.105:23364
 14:38:53,779 INFO

[org.jboss.as.connector.subsystems.datasources.AbstractDataSourceService$AS7DataSourceDeployer]
 (MSC service thread 1-2) IJ020018: Enabling <validate-on-match> for
 java:jboss/datasources/PgDskeycloak
 14:38:53,896 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-1) WFLYJCA0001: Bound data source
 [java:jboss/datasources/KeycloakDS]
 14:38:53,903 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-2) WFLYJCA0001: Bound data source
 [java:jboss/datasources/ExampleDS]
 14:38:53,909 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC
 service thread 1-2) WFLYJCA0001: Bound data source
 [java:jboss/datasources/PgDskeycloak]
 14:38:54,118 INFO  [org.jboss.as.server.deployment] (MSC service thread
 1-2) WFLYSRV0027: Starting deployment of "keycloak-server.war"
 (runtime-name: "keycloak-server.war")
 14:38:54,306 INFO  [org.jboss.ws.common.management] (MSC service thread
 1-1) JBWS022052: Starting JBoss Web Services - Stack CXF Server 5.0.0.Final
 14:38:56,138 INFO  [stdout] (MSC service thread 1-2)
 14:38:56,138 INFO  [stdout] (MSC service thread 1-2)
 -------------------------------------------------------------------
 14:38:56,139 INFO  [stdout] (MSC service thread 1-2) GMS:
 address=ip-10-1-1-245, cluster=ee, physical address=10.1.1.245:7600
 14:38:56,139 INFO  [stdout] (MSC service thread 1-2)
 -------------------------------------------------------------------
 14:38:56,606 INFO
  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (ServerService Thread Pool -- 62) ISPN000078: Starting JGroups channel
 keycloak
 14:38:56,623 INFO
  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (ServerService Thread Pool -- 62) ISPN000094: Received new cluster view for
 channel keycloak: [ip-10-1-1-245|0] (1) [ip-10-1-1-245]
 14:38:56,644 INFO
  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
 (ServerService Thread Pool -- 62) ISPN000079: Channel keycloak local
 address is ip-10-1-1-245, physical addresses are [10.1.1.245:7600]
 14:38:56,651 INFO  [org.infinispan.factories.GlobalComponentRegistry]
 (ServerService Thread Pool -- 62) ISPN000128: Infinispan version:
 Infinispan 'Insanely Bad Elf' 7.2.3.Final
 14:38:57,044 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 65) WFLYCLINF0002: Started users cache from keycloak
 container
 14:38:57,050 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 62) WFLYCLINF0002: Started sessions cache from keycloak
 container
 14:38:57,055 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 63) WFLYCLINF0002: Started realms cache from keycloak
 container
 14:38:57,059 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 64) WFLYCLINF0002: Started loginFailures cache from keycloak
 container
 14:38:58,007 INFO  [org.keycloak.services.resources.KeycloakApplication]
 (ServerService Thread Pool -- 64) Load config from
 /opt/keycloak-1.7.0.Final/standalone/configuration/keycloak-server.json
 14:38:58,755 INFO  [org.hibernate.jpa.internal.util.LogHelper]
 (ServerService Thread Pool -- 64) HHH000204: Processing PersistenceUnitInfo
 [
         name: keycloak-default
         ...]
 14:38:58,812 INFO  [org.hibernate.Version] (ServerService Thread Pool --
 64) HHH000412: Hibernate Core {4.3.10.Final}
 14:38:58,819 INFO  [org.hibernate.cfg.Environment] (ServerService Thread
 Pool -- 64) HHH000206: hibernate.properties not found
 14:38:58,824 INFO  [org.hibernate.cfg.Environment] (ServerService Thread
 Pool -- 64) HHH000021: Bytecode provider name : javassist
 14:38:59,268 INFO  [org.hibernate.annotations.common.Version]
 (ServerService Thread Pool -- 64) HCANN000001: Hibernate Commons
 Annotations {4.0.5.Final}
 14:39:00,264 INFO  [org.hibernate.dialect.Dialect] (ServerService Thread
 Pool -- 64) HHH000400: Using dialect:
 org.hibernate.dialect.PostgreSQL9Dialect
 14:39:00,272 INFO  [org.hibernate.engine.jdbc.internal.LobCreatorBuilder]
 (ServerService Thread Pool -- 64) HHH000424: Disabling contextual LOB
 creation as createClob() method threw error :
 java.lang.reflect.InvocationTargetException
 14:39:00,602 INFO
  [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory] (ServerService
 Thread Pool -- 64) HHH000397: Using ASTQueryTranslatorFactory
 14:39:00,634 INFO  [org.hibernate.validator.internal.util.Version]
 (ServerService Thread Pool -- 64) HV000001: Hibernate Validator 5.1.3.Final
 14:39:04,607 INFO  [org.jboss.as.clustering.infinispan] (ServerService
 Thread Pool -- 64) WFLYCLINF0002: Started offlineSessions cache from
 keycloak container
 14:39:04,665 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Deploying javax.ws.rs.core.Application:
 class org.keycloak.services.resources.KeycloakApplication
 14:39:04,667 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding class resource
 org.keycloak.services.resources.WelcomeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,667 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding class resource
 org.keycloak.services.resources.QRCodeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,668 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding class resource
 org.keycloak.services.resources.JsResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,668 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding class resource
 org.keycloak.services.resources.ThemeResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,668 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding provider singleton
 org.keycloak.services.resources.ModelExceptionMapper from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,669 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding singleton resource
 org.keycloak.services.resources.RealmsResource from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,669 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding provider singleton
 org.keycloak.services.util.ObjectMapperResolver from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,669 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding singleton resource
 org.keycloak.services.resources.ServerVersionResource from Application
 class org.keycloak.services.resources.KeycloakApplication
 14:39:04,669 INFO  [org.jboss.resteasy.spi.ResteasyDeployment]
 (ServerService Thread Pool -- 64) Adding singleton resource
 org.keycloak.services.resources.admin.AdminRoot from Application class
 org.keycloak.services.resources.KeycloakApplication
 14:39:04,757 INFO  [org.wildfly.extension.undertow] (ServerService Thread
 Pool -- 64) WFLYUT0021: Registered web context: /auth
 14:39:04,844 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 61)
 WFLYSRV0010: Deployed "keycloak-server.war" (runtime-name :
 "keycloak-server.war")
 14:39:05,526 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060:
 Http management interface listening on http://10.1.1.245:9990/management
 14:39:05,527 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051:
 Admin console listening on http://10.1.1.245:9990
 14:39:05,531 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025:
 Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) started in 17727ms -
 Started 349 of 613 services (353 services are lazy, passive or on-demand)

 CHARLES-EDOUARD GAGNAIRE
 SysAdmin
 c.gagnaire(a)kreactive.com
 p. 06.27.80.28.53LYON "Le Capitole"
 97, cours Gambetta
 69481 Lyon Cedex 03

 PARIS
 16, rue de Turbigo
 75002 Paris
 [image: Kreactive] <http://www.kreactive.com/>

 [image: Facebook] <https://www.facebook.com/kreactive> [image: Twitter]
 <https://twitter.com/kreactive>

 2015-12-18 15:53 GMT+01:00 Alan Field <afield(a)redhat.com&gt;:

> Hey Charles,
>
> Can you send the full logs and tell me which version of JGroups you are
> using?
>
> Thanks,
> Alan
>
> ------------------------------
>
> *From: *"charles-edouard gagnaire" <c.gagnaire(a)kreactive.com&gt;
> *To: *keycloak-user(a)lists.jboss.org
> *Sent: *Friday, December 18, 2015 9:01:12 AM
> *Subject: *[keycloak-user] Problem running keycloak cluster on EC2 with
> S3_ping
>
>
> hi,
>
> I'm having trouble configuring a Keycloak cluster running on AWS' EC2.
> The database configuration is OK no problem, but i can't manage to get
> the invalidation cache working correctly.
> I configured Infinispan to work with S3_ping plugin (the relevant part of
> my configuration is below).
>
> When i run both server, the connection with the database is Ok, but the
> infinispan logs look like this :
> On Server 1 :
> ...
> 11:00:17,592 INFO  [stdout] (MSC service thread 1-1) GMS:
> address=ip-10-1-7-103, cluster=ee, physical address=10.1.7.103:7600
> ...
> 11:00:18,057 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (ServerService Thread Pool -- 62) ISPN000094: Received new cluster view for
> channel keycloak: [ip-10-1-7-103|0] (1) [ip-10-1-7-103]
> ...
>
> On Server 2 :
> ...
> 11:03:41,159 INFO  [stdout] (MSC service thread 1-1) GMS:
> address=ip-10-1-1-245, cluster=ee, physical address=10.1.1.245:7600
> ...
> 11:03:41,783 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (ServerService Thread Pool -- 62) ISPN000094: Received new cluster view for
> channel keycloak: [ip-10-1-1-245|0] (1) [ip-10-1-1-245]
> ...
>
> In my S3 bucket, i have 2 files created :
> 402ea329-c135-f1e9-2782-02768779e02f.ip-10-1-1-245.list
> a584321f-408b-b2ae-e2dd-d19333db96c4.ip-10-1-7-103.list
>
> And the content of the files is like this :
> File 1 :
> ip-10-1-1-245 402ea329-c135-f1e9-2782-02768779e02f 10.1.1.245:7600 T
>
> File 2 :
> ip-10-1-7-103 a584321f-408b-b2ae-e2dd-d19333db96c4 10.1.7.103:7600 T
>
> When i read the logs, it looks like the infinispan's cache can't contact
> each other.
> I double check my network config, and i tried connecting from one server
> to the other using nc (like this: nc -vvv 10.1.7.103 7600) and this works
> fine.
>
> Is there a way to check the infinispan status of the servers?
> Do you guys got any clue on how to make this works?
>
> Thank you,
> Charles-Edouard
>
> My config looks like this :
>
> - Standalone-ha.xml
> ...
> <datasources>
>                 <driver name="postgresql"
module="org.postgresql">
>
>  <datasource-class>org.postgresql.Driver</datasource-class>
>
> 
<xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class>
>                 </driver>
>                 <datasource
> jndi-name="java:jboss/datasources/PgDskeycloak"
pool-name="PgDskeycloak"
> enabled="true" use-java-context="true">
>
>
<connection-url>jdbc:postgresql://****:5432/keycloak?ApplicationName=keycloak</connection-url>
>                     <driver>postgresql</driver>
>                     <pool>
>                         <min-pool-size>5</min-pool-size>
>                         <initial-pool-size>5</initial-pool-size>
>                         <max-pool-size>100</max-pool-size>
>                         <prefill>true</prefill>
>                     </pool>
>                     <validation>
>                         <valid-connection-checker
>
> 
class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLValidConnectionChecker"></valid-connection-checker>
>                         <exception-sorter
>
> 
class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLExceptionSorter"></exception-sorter>
>                     </validation>
>                     <security>
>                         <user-name>****</user-name>
>                         <password>****</password>
>                     </security>
>                 </datasource>
> ...
>             <stacks default="tcp">
>                 <stack name="udp">
>                     <transport type="UDP"
socket-binding="jgroups-udp"/>
>                     <protocol type="PING"/>
>                     <protocol type="MERGE3"/>
>                     <protocol type="FD_SOCK"
> socket-binding="jgroups-udp-fd"/>
>                     <protocol type="FD_ALL"/>
>                     <protocol type="VERIFY_SUSPECT"/>
>                     <protocol type="pbcast.NAKACK2"/>
>                     <protocol type="UNICAST3"/>
>                     <protocol type="pbcast.STABLE"/>
>                     <protocol type="pbcast.GMS"/>
>                     <protocol type="UFC"/>
>                     <protocol type="MFC"/>
>                     <protocol type="FRAG2"/>
>                     <protocol type="RSVP"/>
>                 </stack>
>                 <stack name="tcp">
>                     <transport type="TCP"
socket-binding="jgroups-tcp"/>
>                     <protocol type="S3_PING" >
>                         <property
name="location">****</property>
>                         <property
name="access_key">****</property>
>                         <property
name="secret_access_key">****</property>
>                     </protocol>
>                     <!-- <protocol type="MPING"
> socket-binding="jgroups-mping"/> -->
>                     <protocol type="MERGE3"/>
>                     <protocol type="FD_SOCK"
> socket-binding="jgroups-tcp-fd"/>
>                     <protocol type="FD"/>
>                     <protocol type="VERIFY_SUSPECT"/>
>                     <protocol type="pbcast.NAKACK2"/>
>                     <protocol type="UNICAST3"/>
>                     <protocol type="pbcast.STABLE"/>
>                     <protocol type="pbcast.GMS"/>
>                     <protocol type="MFC"/>
>                     <protocol type="FRAG2"/>
>                     <protocol type="RSVP"/>
>                 </stack>
> ...
>     <interfaces>
>         <interface name="management">
>                 <nic name="eth0"/>
>         </interface>
>         <interface name="public">
>             <nic name="eth0"/>
>         </interface>
>         <!-- TODO - only show this if the jacorb subsystem is added  -->
>         <interface name="unsecure">
>             <!--
>               ~  Used for IIOP sockets in the standard configuration.
>               ~                  To secure JacORB you need to setup SSL
>               -->
>             <nic name="eth0"/>
>         </interface>
>     </interfaces>
>
> - keycloak-server.json
> {
>     "providers": [
>         "classpath:${jboss.server.config.dir}/providers/*"
>     ],
>
>     "admin": {
>         "realm": "master"
>     },
>
>     "eventsStore": {
>         "provider": "jpa",
>         "jpa": {
>             "exclude-events": [ "REFRESH_TOKEN" ]
>         }
>     },
>
>     "realm": {
>         "provider": "jpa"
>     },
>
>     "user": {
>         "provider": "jpa"
>     },
>
>     "userSessionPersister": {
>         "provider": "jpa"
>     },
>
>     "timer": {
>         "provider": "basic"
>     },
>
>     "theme": {
>         "default": "keycloak",
>         "staticMaxAge": 2592000,
>         "cacheTemplates": true,
>         "cacheThemes": true,
>         "folder": {
>           "dir": "${jboss.server.config.dir}/themes"
>         }
>     },
>
>     "scheduled": {
>         "interval": 900
>     },
>
>     "connectionsHttpClient": {
>         "default": {
>             "disable-trust-manager": true
>         }
>     },
>
>     "connectionsJpa": {
>         "default": {
>             "dataSource": "java:jboss/datasources/PgDskeycloak",
>             "databaseSchema": "update"
>         }
>     },
>
>     "connectionsInfinispan": {
>         "default" : {
>             "cacheContainer" : "java:jboss/infinispan/Keycloak"
>         }
>     }
> }
>
> CHARLES-EDOUARD GAGNAIRE
> SysAdmin
> c.gagnaire(a)kreactive.com
> p. 06.27.80.28.53LYON "Le Capitole"
> 97, cours Gambetta
> 69481 Lyon Cedex 03
>
> PARIS
> 16, rue de Turbigo
> 75002 Paris
> [image: Kreactive] <http://www.kreactive.com/>
>
>
> [image: Facebook] <https://www.facebook.com/kreactive> [image: Twitter]
> <https://twitter.com/kreactive>
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
>
>

 _______________________________________________
 keycloak-user mailing list
 keycloak-user(a)lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/keycloak-user

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

Re: [keycloak-user] Problem running keycloak cluster on EC2 with S3_ping