[keycloak-user] Problem running keycloak cluster on EC2 with S3_ping

Alan Field afield at redhat.com
Fri Dec 18 10:21:54 EST 2015


Hey Charles, 

Thanks for the logs. I'm not sure what is wrong, but it looks like each server is creating a cluster of 1. I'll try it with my AWS account to see if I can figure out what is wrong. 

Alan 

----- Original Message -----

> From: "charles-edouard gagnaire" <c.gagnaire at kreactive.com>
> To: "Alan Field" <afield at redhat.com>
> Cc: keycloak-user at lists.jboss.org
> Sent: Friday, December 18, 2015 10:04:53 AM
> Subject: Re: [keycloak-user] Problem running keycloak cluster on EC2 with
> S3_ping

> First i want to thank you guys for the quick answers, i was still reading the
> "Replace use of Infinispan with User Sessions SPI ?" discussion.

> Yes of course i can send all the logs. You'll find them below.

> The JGroups version is the one shipping with keycloak 1.7, but the problem
> was the same with Keycloak 1.6.
> Looking at the config file, it looks like i'm using : <subsystem
> xmlns="urn:jboss:domain:jgroups:3.0">

> I didn't mention it but i use the archive i found on Keycloak website. The
> archive is "keycloak-1.7.0.Final.tar.gz".
> I just untar and modify the config files, then i launched it using :
> /opt/keycloak-1.7.0.Final/bin/standalone.sh -c standalone-ha.xml

> Thank you again for your help

> The logs for server 1 are :
> =========================================================================

> JBoss Bootstrap Environment

> JBOSS_HOME: /opt/keycloak-1.7.0.Final

> JAVA: /usr/lib/jvm/jre/bin/java

> JAVA_OPTS: -server -XX:+UseCompressedOops -server -XX:+UseCompressedOops
> -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true
> -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

> =========================================================================

> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support
> was removed in 8.0
> 14:38:44,910 INFO [org.jboss.modules] (main) JBoss Modules version
> 1.4.3.Final
> 14:38:45,091 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> 14:38:45,163 INFO [ org.jboss.as ] (MSC service thread 1-2) WFLYSRV0049:
> Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) starting
> 14:38:46,358 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 24) WFLYCTL0028: Attribute
> 'job-repository-type' in the resource at address '/subsystem=batch' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:46,360 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'enabled' in the
> resource at address '/subsystem=datasources/data-source=PgDskeycloak' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:46,362 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'enabled' in the
> resource at address '/subsystem=datasources/data-source=ExampleDS' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:46,362 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'enabled' in the
> resource at address '/subsystem=datasources/data-source=KeycloakDS' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:46,370 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 21) WFLYCTL0028: Attribute 'default-stack' in
> the resource at address '/subsystem=jgroups' is deprecated, and may be
> removed in future version. See the attribute description in the output of
> the read-resource-description operation to learn more about the deprecation.
> 14:38:46,572 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039:
> Creating http management service using socket-binding (management-http)
> 14:38:46,589 INFO [org.xnio] (MSC service thread 1-4) XNIO version
> 3.3.1.Final
> 14:38:46,607 INFO [org.xnio.nio] (MSC service thread 1-4) XNIO NIO
> Implementation Version 3.3.1.Final
> 14:38:46,655 INFO [org.jboss.remoting] (MSC service thread 1-4) JBoss
> Remoting version 4.0.9.Final
> 14:38:46,687 INFO [ org.wildfly.extension.io ] (ServerService Thread Pool --
> 38) WFLYIO001: Worker 'default' has auto-configured to 4 core threads with
> 32 task threads based on your 2 available processors
> 14:38:46,685 INFO [org.jboss.as.connector.subsystems.datasources]
> (ServerService Thread Pool -- 35) WFLYJCA0005: Deploying non-JDBC-compliant
> driver class org.postgresql.Driver (version 9.4)
> 14:38:46,715 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread
> Pool -- 43) WFLYCLJG0001: Activating JGroups subsystem.
> 14:38:46,724 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 39) WFLYCLINF0001: Activating Infinispan subsystem.
> 14:38:46,744 INFO [org.jboss.as.connector] (MSC service thread 1-4)
> WFLYJCA0009: Starting JCA Subsystem (IronJacamar 1.2.5.Final)
> 14:38:46,746 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread
> 1-2) WFLYJCA0018: Started Driver service with driver-name = postgresql
> 14:38:46,767 INFO [org.jboss.as.connector.subsystems.datasources]
> (ServerService Thread Pool -- 35) WFLYJCA0004: Deploying JDBC-compliant
> driver class org.h2.Driver (version 1.3)
> 14:38:46,769 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread
> 1-2) WFLYJCA0018: Started Driver service with driver-name = h2
> 14:38:46,781 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 46)
> WFLYJSF0007: Activated the following JSF Implementations: [main]
> 14:38:46,772 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 49)
> WFLYNAM0001: Activating Naming Subsystem
> 14:38:46,914 INFO [org.jboss.as.security] (ServerService Thread Pool -- 56)
> WFLYSEC0002: Activating Security Subsystem
> 14:38:46,916 INFO [org.jboss.as.security] (MSC service thread 1-1)
> WFLYSEC0001: Current PicketBox version=4.9.2.Final
> 14:38:46,932 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 57)
> WFLYTX0013: Node identifier property is set to the default value. Please
> make sure it is unique.
> 14:38:46,957 INFO [org.jboss.as.webservices] (ServerService Thread Pool --
> 59) WFLYWS0002: Activating WebServices Extension
> 14:38:46,985 INFO [org.jboss.as.naming] (MSC service thread 1-2) WFLYNAM0003:
> Starting Naming Service
> 14:38:46,992 INFO [org.jboss.as.mail.extension] (MSC service thread 1-4)
> WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
> 14:38:47,115 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1)
> WFLYUT0003: Undertow 1.2.9.Final starting
> 14:38:47,119 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool
> -- 58) WFLYUT0003: Undertow 1.2.9.Final starting
> 14:38:47,206 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool
> -- 58) WFLYUT0014: Creating file handler for path
> /opt/keycloak-1.7.0.Final/welcome-content
> 14:38:47,229 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4)
> WFLYUT0012: Started server default-server.
> 14:38:47,263 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4)
> WFLYUT0018: Host default-host starting
> 14:38:47,320 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2)
> WFLYUT0006: Undertow AJP listener ajp listening on / 10.1.7.103:8009
> 14:38:47,324 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3)
> WFLYUT0006: Undertow HTTP listener default listening on / 10.1.7.103:8080
> 14:38:47,339 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 62)
> MODCLUSTER000001: Initializing mod_cluster version 1.3.1.Final
> 14:38:47,372 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 62)
> MODCLUSTER000032: Listening to proxy advertisements on / 224.0.1.105:23364
> 14:38:47,478 INFO
> [org.jboss.as.connector.subsystems.datasources.AbstractDataSourceService$AS7DataSourceDeployer]
> (MSC service thread 1-1) IJ020018: Enabling <validate-on-match> for
> java:jboss/datasources/PgDskeycloak
> 14:38:47,513 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
> service thread 1-2) WFLYJCA0001: Bound data source
> [java:jboss/datasources/KeycloakDS]
> 14:38:47,513 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
> service thread 1-2) WFLYJCA0001: Bound data source
> [java:jboss/datasources/ExampleDS]
> 14:38:47,530 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
> service thread 1-2) WFLYJCA0001: Bound data source
> [java:jboss/datasources/PgDskeycloak]
> 14:38:47,673 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4)
> WFLYSRV0027: Starting deployment of "keycloak-server.war" (runtime-name:
> "keycloak-server.war")
> 14:38:47,820 INFO [org.jboss.ws.common.management] (MSC service thread 1-3)
> JBWS022052: Starting JBoss Web Services - Stack CXF Server 5.0.0.Final
> 14:38:48,898 INFO [stdout] (MSC service thread 1-2)
> 14:38:48,898 INFO [stdout] (MSC service thread 1-2)
> -------------------------------------------------------------------
> 14:38:48,898 INFO [stdout] (MSC service thread 1-2) GMS:
> address=ip-10-1-7-103, cluster=ee, physical address= 10.1.7.103:7600
> 14:38:48,899 INFO [stdout] (MSC service thread 1-2)
> -------------------------------------------------------------------
> 14:38:49,250 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
> Thread Pool -- 62) ISPN000078: Starting JGroups channel keycloak
> 14:38:49,265 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
> Thread Pool -- 62) ISPN000094: Received new cluster view for channel
> keycloak: [ip-10-1-7-103|0] (1) [ip-10-1-7-103]
> 14:38:49,273 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
> Thread Pool -- 62) ISPN000079: Channel keycloak local address is
> ip-10-1-7-103, physical addresses are [ 10.1.7.103:7600 ]
> 14:38:49,277 INFO [org.infinispan.factories.GlobalComponentRegistry]
> (ServerService Thread Pool -- 62) ISPN000128: Infinispan version: Infinispan
> 'Insanely Bad Elf' 7.2.3.Final
> 14:38:49,521 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 65) WFLYCLINF0002: Started users cache from keycloak container
> 14:38:49,529 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 62) WFLYCLINF0002: Started loginFailures cache from keycloak
> container
> 14:38:49,530 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 66) WFLYCLINF0002: Started sessions cache from keycloak container
> 14:38:49,536 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 63) WFLYCLINF0002: Started realms cache from keycloak container
> 14:38:50,116 INFO [org.keycloak.services.resources.KeycloakApplication]
> (ServerService Thread Pool -- 66) Load config from
> /opt/keycloak-1.7.0.Final/standalone/configuration/keycloak-server.json
> 14:38:50,638 INFO [org.hibernate.jpa.internal.util.LogHelper] (ServerService
> Thread Pool -- 66) HHH000204: Processing PersistenceUnitInfo [
> name: keycloak-default
> ...]
> 14:38:50,690 INFO [org.hibernate.Version] (ServerService Thread Pool -- 66)
> HHH000412: Hibernate Core {4.3.10.Final}
> 14:38:50,691 INFO [org.hibernate.cfg.Environment] (ServerService Thread Pool
> -- 66) HHH000206: hibernate.properties not found
> 14:38:50,693 INFO [org.hibernate.cfg.Environment] (ServerService Thread Pool
> -- 66) HHH000021: Bytecode provider name : javassist
> 14:38:50,842 INFO [org.hibernate.annotations.common.Version] (ServerService
> Thread Pool -- 66) HCANN000001: Hibernate Commons Annotations {4.0.5.Final}
> 14:38:51,794 INFO [org.hibernate.dialect.Dialect] (ServerService Thread Pool
> -- 66) HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL9Dialect
> 14:38:51,803 INFO [org.hibernate.engine.jdbc.internal.LobCreatorBuilder]
> (ServerService Thread Pool -- 66) HHH000424: Disabling contextual LOB
> creation as createClob() method threw error :
> java.lang.reflect.InvocationTargetException
> 14:38:52,120 INFO [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory]
> (ServerService Thread Pool -- 66) HHH000397: Using ASTQueryTranslatorFactory
> 14:38:52,156 INFO [org.hibernate.validator.internal.util.Version]
> (ServerService Thread Pool -- 66) HV000001: Hibernate Validator 5.1.3.Final
> 14:38:53,706 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 66) WFLYCLINF0002: Started offlineSessions cache from keycloak
> container
> 14:38:53,748 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Deploying javax.ws.rs.core.Application: class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,750 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding class resource
> org.keycloak.services.resources.WelcomeResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,750 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding class resource
> org.keycloak.services.resources.JsResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,750 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding class resource
> org.keycloak.services.resources.QRCodeResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,750 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding class resource
> org.keycloak.services.resources.ThemeResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,751 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding singleton resource
> org.keycloak.services.resources.RealmsResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,751 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding singleton resource
> org.keycloak.services.resources.ServerVersionResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,751 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding singleton resource
> org.keycloak.services.resources.admin.AdminRoot from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,751 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding provider singleton
> org.keycloak.services.util.ObjectMapperResolver from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,752 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 66) Adding provider singleton
> org.keycloak.services.resources.ModelExceptionMapper from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:38:53,824 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool
> -- 66) WFLYUT0021: Registered web context: /auth
> 14:38:53,920 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61)
> WFLYSRV0010: Deployed "keycloak-server.war" (runtime-name :
> "keycloak-server.war")
> 14:38:54,021 INFO [ org.jboss.as ] (Controller Boot Thread) WFLYSRV0060: Http
> management interface listening on http://10.1.7.103:9990/management
> 14:38:54,021 INFO [ org.jboss.as ] (Controller Boot Thread) WFLYSRV0051:
> Admin console listening on http://10.1.7.103:9990
> 14:38:54,022 INFO [ org.jboss.as ] (Controller Boot Thread) WFLYSRV0025:
> Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) started in 9388ms - Started
> 349 of 613 services (353 services are lazy, passive or on-demand)

> The logs for server 2 are :
> =========================================================================

> JBoss Bootstrap Environment

> JBOSS_HOME: /opt/keycloak-1.7.0.Final

> JAVA: /usr/lib/jvm/jre/bin/java

> JAVA_OPTS: -server -XX:+UseCompressedOops -server -XX:+UseCompressedOops
> -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true
> -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

> =========================================================================

> 14:38:48,239 INFO [org.jboss.modules] (main) JBoss Modules version
> 1.4.3.Final
> 14:38:48,723 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
> 14:38:48,896 INFO [ org.jboss.as ] (MSC service thread 1-2) WFLYSRV0049:
> Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) starting
> 14:38:50,979 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 8) WFLYCTL0028: Attribute
> 'job-repository-type' in the resource at address '/subsystem=batch' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:50,983 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 11) WFLYCTL0028: Attribute 'enabled' in the
> resource at address '/subsystem=datasources/data-source=PgDskeycloak' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:50,986 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 11) WFLYCTL0028: Attribute 'enabled' in the
> resource at address '/subsystem=datasources/data-source=ExampleDS' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:51,010 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 14) WFLYCTL0028: Attribute 'default-stack' in
> the resource at address '/subsystem=jgroups' is deprecated, and may be
> removed in future version. See the attribute description in the output of
> the read-resource-description operation to learn more about the deprecation.
> 14:38:51,044 INFO [org.jboss.as.controller.management-deprecated]
> (ServerService Thread Pool -- 11) WFLYCTL0028: Attribute 'enabled' in the
> resource at address '/subsystem=datasources/data-source=KeycloakDS' is
> deprecated, and may be removed in future version. See the attribute
> description in the output of the read-resource-description operation to
> learn more about the deprecation.
> 14:38:51,452 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039:
> Creating http management service using socket-binding (management-http)
> 14:38:51,499 INFO [org.xnio] (MSC service thread 1-1) XNIO version
> 3.3.1.Final
> 14:38:51,520 INFO [org.xnio.nio] (MSC service thread 1-1) XNIO NIO
> Implementation Version 3.3.1.Final
> 14:38:51,590 INFO [org.jboss.as.connector.subsystems.datasources]
> (ServerService Thread Pool -- 35) WFLYJCA0005: Deploying non-JDBC-compliant
> driver class org.postgresql.Driver (version 9.4)
> 14:38:51,603 INFO [ org.wildfly.extension.io ] (ServerService Thread Pool --
> 38) WFLYIO001: Worker 'default' has auto-configured to 2 core threads with
> 16 task threads based on your 1 available processors
> 14:38:51,601 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 39) WFLYCLINF0001: Activating Infinispan subsystem.
> 14:38:51,634 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread
> Pool -- 43) WFLYCLJG0001: Activating JGroups subsystem.
> 14:38:51,694 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 49)
> WFLYNAM0001: Activating Naming Subsystem
> 14:38:51,666 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 46)
> WFLYJSF0007: Activated the following JSF Implementations: [main]
> 14:38:51,696 INFO [org.jboss.as.connector] (MSC service thread 1-2)
> WFLYJCA0009: Starting JCA Subsystem (IronJacamar 1.2.5.Final)
> 14:38:51,932 INFO [org.jboss.as.webservices] (ServerService Thread Pool --
> 59) WFLYWS0002: Activating WebServices Extension
> 14:38:51,970 INFO [org.jboss.remoting] (MSC service thread 1-1) JBoss
> Remoting version 4.0.9.Final
> 14:38:51,975 INFO [org.jboss.as.security] (ServerService Thread Pool -- 56)
> WFLYSEC0002: Activating Security Subsystem
> 14:38:51,972 INFO [org.jboss.as.connector.subsystems.datasources]
> (ServerService Thread Pool -- 35) WFLYJCA0004: Deploying JDBC-compliant
> driver class org.h2.Driver (version 1.3)
> 14:38:51,971 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 57)
> WFLYTX0013: Node identifier property is set to the default value. Please
> make sure it is unique.
> 14:38:52,140 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool
> -- 58) WFLYUT0003: Undertow 1.2.9.Final starting
> 14:38:52,187 INFO [org.jboss.as.security] (MSC service thread 1-2)
> WFLYSEC0001: Current PicketBox version=4.9.2.Final
> 14:38:52,224 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread
> 1-1) WFLYJCA0018: Started Driver service with driver-name = postgresql
> 14:38:52,225 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread
> 1-1) WFLYJCA0018: Started Driver service with driver-name = h2
> 14:38:52,368 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1)
> WFLYUT0003: Undertow 1.2.9.Final starting
> 14:38:52,369 INFO [org.jboss.as.naming] (MSC service thread 1-2) WFLYNAM0003:
> Starting Naming Service
> 14:38:52,471 INFO [org.jboss.as.mail.extension] (MSC service thread 1-2)
> WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
> 14:38:52,710 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool
> -- 58) WFLYUT0014: Creating file handler for path
> /opt/keycloak-1.7.0.Final/welcome-content
> 14:38:52,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2)
> WFLYUT0012: Started server default-server.
> 14:38:53,133 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2)
> WFLYUT0006: Undertow HTTP listener default listening on / 10.1.1.245:8080
> 14:38:53,166 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2)
> WFLYUT0018: Host default-host starting
> 14:38:53,192 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2)
> WFLYUT0006: Undertow AJP listener ajp listening on / 10.1.1.245:8009
> 14:38:53,211 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 62)
> MODCLUSTER000001: Initializing mod_cluster version 1.3.1.Final
> 14:38:53,307 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 62)
> MODCLUSTER000032: Listening to proxy advertisements on / 224.0.1.105:23364
> 14:38:53,779 INFO
> [org.jboss.as.connector.subsystems.datasources.AbstractDataSourceService$AS7DataSourceDeployer]
> (MSC service thread 1-2) IJ020018: Enabling <validate-on-match> for
> java:jboss/datasources/PgDskeycloak
> 14:38:53,896 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
> service thread 1-1) WFLYJCA0001: Bound data source
> [java:jboss/datasources/KeycloakDS]
> 14:38:53,903 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
> service thread 1-2) WFLYJCA0001: Bound data source
> [java:jboss/datasources/ExampleDS]
> 14:38:53,909 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
> service thread 1-2) WFLYJCA0001: Bound data source
> [java:jboss/datasources/PgDskeycloak]
> 14:38:54,118 INFO [org.jboss.as.server.deployment] (MSC service thread 1-2)
> WFLYSRV0027: Starting deployment of "keycloak-server.war" (runtime-name:
> "keycloak-server.war")
> 14:38:54,306 INFO [org.jboss.ws.common.management] (MSC service thread 1-1)
> JBWS022052: Starting JBoss Web Services - Stack CXF Server 5.0.0.Final
> 14:38:56,138 INFO [stdout] (MSC service thread 1-2)
> 14:38:56,138 INFO [stdout] (MSC service thread 1-2)
> -------------------------------------------------------------------
> 14:38:56,139 INFO [stdout] (MSC service thread 1-2) GMS:
> address=ip-10-1-1-245, cluster=ee, physical address= 10.1.1.245:7600
> 14:38:56,139 INFO [stdout] (MSC service thread 1-2)
> -------------------------------------------------------------------
> 14:38:56,606 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
> Thread Pool -- 62) ISPN000078: Starting JGroups channel keycloak
> 14:38:56,623 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
> Thread Pool -- 62) ISPN000094: Received new cluster view for channel
> keycloak: [ip-10-1-1-245|0] (1) [ip-10-1-1-245]
> 14:38:56,644 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService
> Thread Pool -- 62) ISPN000079: Channel keycloak local address is
> ip-10-1-1-245, physical addresses are [ 10.1.1.245:7600 ]
> 14:38:56,651 INFO [org.infinispan.factories.GlobalComponentRegistry]
> (ServerService Thread Pool -- 62) ISPN000128: Infinispan version: Infinispan
> 'Insanely Bad Elf' 7.2.3.Final
> 14:38:57,044 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 65) WFLYCLINF0002: Started users cache from keycloak container
> 14:38:57,050 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 62) WFLYCLINF0002: Started sessions cache from keycloak container
> 14:38:57,055 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 63) WFLYCLINF0002: Started realms cache from keycloak container
> 14:38:57,059 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 64) WFLYCLINF0002: Started loginFailures cache from keycloak
> container
> 14:38:58,007 INFO [org.keycloak.services.resources.KeycloakApplication]
> (ServerService Thread Pool -- 64) Load config from
> /opt/keycloak-1.7.0.Final/standalone/configuration/keycloak-server.json
> 14:38:58,755 INFO [org.hibernate.jpa.internal.util.LogHelper] (ServerService
> Thread Pool -- 64) HHH000204: Processing PersistenceUnitInfo [
> name: keycloak-default
> ...]
> 14:38:58,812 INFO [org.hibernate.Version] (ServerService Thread Pool -- 64)
> HHH000412: Hibernate Core {4.3.10.Final}
> 14:38:58,819 INFO [org.hibernate.cfg.Environment] (ServerService Thread Pool
> -- 64) HHH000206: hibernate.properties not found
> 14:38:58,824 INFO [org.hibernate.cfg.Environment] (ServerService Thread Pool
> -- 64) HHH000021: Bytecode provider name : javassist
> 14:38:59,268 INFO [org.hibernate.annotations.common.Version] (ServerService
> Thread Pool -- 64) HCANN000001: Hibernate Commons Annotations {4.0.5.Final}
> 14:39:00,264 INFO [org.hibernate.dialect.Dialect] (ServerService Thread Pool
> -- 64) HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL9Dialect
> 14:39:00,272 INFO [org.hibernate.engine.jdbc.internal.LobCreatorBuilder]
> (ServerService Thread Pool -- 64) HHH000424: Disabling contextual LOB
> creation as createClob() method threw error :
> java.lang.reflect.InvocationTargetException
> 14:39:00,602 INFO [org.hibernate.hql.internal.ast.ASTQueryTranslatorFactory]
> (ServerService Thread Pool -- 64) HHH000397: Using ASTQueryTranslatorFactory
> 14:39:00,634 INFO [org.hibernate.validator.internal.util.Version]
> (ServerService Thread Pool -- 64) HV000001: Hibernate Validator 5.1.3.Final
> 14:39:04,607 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread
> Pool -- 64) WFLYCLINF0002: Started offlineSessions cache from keycloak
> container
> 14:39:04,665 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Deploying javax.ws.rs.core.Application: class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,667 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding class resource
> org.keycloak.services.resources.WelcomeResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,667 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding class resource
> org.keycloak.services.resources.QRCodeResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,668 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding class resource
> org.keycloak.services.resources.JsResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,668 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding class resource
> org.keycloak.services.resources.ThemeResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,668 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding provider singleton
> org.keycloak.services.resources.ModelExceptionMapper from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,669 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding singleton resource
> org.keycloak.services.resources.RealmsResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,669 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding provider singleton
> org.keycloak.services.util.ObjectMapperResolver from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,669 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding singleton resource
> org.keycloak.services.resources.ServerVersionResource from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,669 INFO [org.jboss.resteasy.spi.ResteasyDeployment] (ServerService
> Thread Pool -- 64) Adding singleton resource
> org.keycloak.services.resources.admin.AdminRoot from Application class
> org.keycloak.services.resources.KeycloakApplication
> 14:39:04,757 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool
> -- 64) WFLYUT0021: Registered web context: /auth
> 14:39:04,844 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61)
> WFLYSRV0010: Deployed "keycloak-server.war" (runtime-name :
> "keycloak-server.war")
> 14:39:05,526 INFO [ org.jboss.as ] (Controller Boot Thread) WFLYSRV0060: Http
> management interface listening on http://10.1.1.245:9990/management
> 14:39:05,527 INFO [ org.jboss.as ] (Controller Boot Thread) WFLYSRV0051:
> Admin console listening on http://10.1.1.245:9990
> 14:39:05,531 INFO [ org.jboss.as ] (Controller Boot Thread) WFLYSRV0025:
> Keycloak 1.7.0.Final (WildFly Core 1.0.2.Final) started in 17727ms - Started
> 349 of 613 services (353 services are lazy, passive or on-demand)

> CHARLES-EDOUARD GAGNAIRE
> SysAdmin
> c.gagnaire at kreactive.com
> p. 06.27.80.28.53
> LYON "Le Capitole"
> 97, cours Gambetta
> 69481 Lyon Cedex 03

> PARIS
> 16, rue de Turbigo
> 75002 Paris

> 

> 2015-12-18 15:53 GMT+01:00 Alan Field < afield at redhat.com > :

> > Hey Charles,
> 

> > Can you send the full logs and tell me which version of JGroups you are
> > using?
> 

> > Thanks,
> 
> > Alan
> 

> > > From: "charles-edouard gagnaire" < c.gagnaire at kreactive.com >
> > 
> 
> > > To: keycloak-user at lists.jboss.org
> > 
> 
> > > Sent: Friday, December 18, 2015 9:01:12 AM
> > 
> 
> > > Subject: [keycloak-user] Problem running keycloak cluster on EC2 with
> > > S3_ping
> > 
> 

> > > hi,
> > 
> 

> > > I'm having trouble configuring a Keycloak cluster running on AWS' EC2.
> > 
> 
> > > The database configuration is OK no problem, but i can't manage to get
> > > the
> > > invalidation cache working correctly.
> > 
> 
> > > I configured Infinispan to work with S3_ping plugin (the relevant part of
> > > my
> > > configuration is below).
> > 
> 

> > > When i run both server, the connection with the database is Ok, but the
> > > infinispan logs look like this :
> > 
> 
> > > On Server 1 :
> > 
> 
> > > ...
> > 
> 
> > > 11:00:17,592 INFO [stdout] (MSC service thread 1-1) GMS:
> > > address=ip-10-1-7-103, cluster=ee, physical address= 10.1.7.103:7600
> > 
> 
> > > ...
> > 
> 
> > > 11:00:18,057 INFO
> > > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > > (ServerService
> > > Thread Pool -- 62) ISPN000094: Received new cluster view for channel
> > > keycloak: [ip-10-1-7-103|0] (1) [ip-10-1-7-103]
> > 
> 
> > > ...
> > 
> 

> > > On Server 2 :
> > 
> 
> > > ...
> > 
> 
> > > 11:03:41,159 INFO [stdout] (MSC service thread 1-1) GMS:
> > > address=ip-10-1-1-245, cluster=ee, physical address= 10.1.1.245:7600
> > 
> 
> > > ...
> > 
> 
> > > 11:03:41,783 INFO
> > > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > > (ServerService
> > > Thread Pool -- 62) ISPN000094: Received new cluster view for channel
> > > keycloak: [ip-10-1-1-245|0] (1) [ip-10-1-1-245]
> > 
> 
> > > ...
> > 
> 

> > > In my S3 bucket, i have 2 files created :
> > 
> 
> > > 402ea329-c135-f1e9-2782-02768779e02f.ip-10-1-1-245.list
> > 
> 
> > > a584321f-408b-b2ae-e2dd-d19333db96c4.ip-10-1-7-103.list
> > 
> 

> > > And the content of the files is like this :
> > 
> 
> > > File 1 :
> > 
> 
> > > ip-10-1-1-245 402ea329-c135-f1e9-2782-02768779e02f 10.1.1.245:7600 T
> > 
> 

> > > File 2 :
> > 
> 
> > > ip-10-1-7-103 a584321f-408b-b2ae-e2dd-d19333db96c4 10.1.7.103:7600 T
> > 
> 

> > > When i read the logs, it looks like the infinispan's cache can't contact
> > > each
> > > other.
> > 
> 
> > > I double check my network config, and i tried connecting from one server
> > > to
> > > the other using nc (like this: nc -vvv 10.1.7.103 7600) and this works
> > > fine.
> > 
> 

> > > Is there a way to check the infinispan status of the servers?
> > 
> 
> > > Do you guys got any clue on how to make this works?
> > 
> 

> > > Thank you,
> > 
> 
> > > Charles-Edouard
> > 
> 

> > > My config looks like this :
> > 
> 

> > > - Standalone-ha.xml
> > 
> 
> > > ...
> > 
> 
> > > <datasources>
> > 
> 
> > > <driver name="postgresql" module="org.postgresql">
> > 
> 
> > > <datasource-class>org.postgresql.Driver</datasource-class>
> > 
> 
> > > <xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class>
> > 
> 
> > > </driver>
> > 
> 
> > > <datasource jndi-name="java:jboss/datasources/PgDskeycloak"
> > > pool-name="PgDskeycloak" enabled="true" use-java-context="true">
> > 
> 
> > > <connection-url>jdbc:postgresql://****:5432/keycloak?ApplicationName=keycloak</connection-url>
> > 
> 
> > > <driver>postgresql</driver>
> > 
> 
> > > <pool>
> > 
> 
> > > <min-pool-size>5</min-pool-size>
> > 
> 
> > > <initial-pool-size>5</initial-pool-size>
> > 
> 
> > > <max-pool-size>100</max-pool-size>
> > 
> 
> > > <prefill>true</prefill>
> > 
> 
> > > </pool>
> > 
> 
> > > <validation>
> > 
> 
> > > <valid-connection-checker
> > 
> 
> > > class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLValidConnectionChecker"></valid-connection-checker>
> > 
> 
> > > <exception-sorter
> > 
> 
> > > class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLExceptionSorter"></exception-sorter>
> > 
> 
> > > </validation>
> > 
> 
> > > <security>
> > 
> 
> > > <user-name>****</user-name>
> > 
> 
> > > <password>****</password>
> > 
> 
> > > </security>
> > 
> 
> > > </datasource>
> > 
> 
> > > ...
> > 
> 
> > > <stacks default="tcp">
> > 
> 
> > > <stack name="udp">
> > 
> 
> > > <transport type="UDP" socket-binding="jgroups-udp"/>
> > 
> 
> > > <protocol type="PING"/>
> > 
> 
> > > <protocol type="MERGE3"/>
> > 
> 
> > > <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
> > 
> 
> > > <protocol type="FD_ALL"/>
> > 
> 
> > > <protocol type="VERIFY_SUSPECT"/>
> > 
> 
> > > <protocol type="pbcast.NAKACK2"/>
> > 
> 
> > > <protocol type="UNICAST3"/>
> > 
> 
> > > <protocol type="pbcast.STABLE"/>
> > 
> 
> > > <protocol type="pbcast.GMS"/>
> > 
> 
> > > <protocol type="UFC"/>
> > 
> 
> > > <protocol type="MFC"/>
> > 
> 
> > > <protocol type="FRAG2"/>
> > 
> 
> > > <protocol type="RSVP"/>
> > 
> 
> > > </stack>
> > 
> 
> > > <stack name="tcp">
> > 
> 
> > > <transport type="TCP" socket-binding="jgroups-tcp"/>
> > 
> 
> > > <protocol type="S3_PING" >
> > 
> 
> > > <property name="location">****</property>
> > 
> 
> > > <property name="access_key">****</property>
> > 
> 
> > > <property name="secret_access_key">****</property>
> > 
> 
> > > </protocol>
> > 
> 
> > > <!-- <protocol type="MPING" socket-binding="jgroups-mping"/> -->
> > 
> 
> > > <protocol type="MERGE3"/>
> > 
> 
> > > <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
> > 
> 
> > > <protocol type="FD"/>
> > 
> 
> > > <protocol type="VERIFY_SUSPECT"/>
> > 
> 
> > > <protocol type="pbcast.NAKACK2"/>
> > 
> 
> > > <protocol type="UNICAST3"/>
> > 
> 
> > > <protocol type="pbcast.STABLE"/>
> > 
> 
> > > <protocol type="pbcast.GMS"/>
> > 
> 
> > > <protocol type="MFC"/>
> > 
> 
> > > <protocol type="FRAG2"/>
> > 
> 
> > > <protocol type="RSVP"/>
> > 
> 
> > > </stack>
> > 
> 
> > > ...
> > 
> 
> > > <interfaces>
> > 
> 
> > > <interface name="management">
> > 
> 
> > > <nic name="eth0"/>
> > 
> 
> > > </interface>
> > 
> 
> > > <interface name="public">
> > 
> 
> > > <nic name="eth0"/>
> > 
> 
> > > </interface>
> > 
> 
> > > <!-- TODO - only show this if the jacorb subsystem is added -->
> > 
> 
> > > <interface name="unsecure">
> > 
> 
> > > <!--
> > 
> 
> > > ~ Used for IIOP sockets in the standard configuration.
> > 
> 
> > > ~ To secure JacORB you need to setup SSL
> > 
> 
> > > -->
> > 
> 
> > > <nic name="eth0"/>
> > 
> 
> > > </interface>
> > 
> 
> > > </interfaces>
> > 
> 

> > > - keycloak-server.json
> > 
> 
> > > {
> > 
> 
> > > "providers": [
> > 
> 
> > > "classpath:${jboss.server.config.dir}/providers/*"
> > 
> 
> > > ],
> > 
> 

> > > "admin": {
> > 
> 
> > > "realm": "master"
> > 
> 
> > > },
> > 
> 

> > > "eventsStore": {
> > 
> 
> > > "provider": "jpa",
> > 
> 
> > > "jpa": {
> > 
> 
> > > "exclude-events": [ "REFRESH_TOKEN" ]
> > 
> 
> > > }
> > 
> 
> > > },
> > 
> 

> > > "realm": {
> > 
> 
> > > "provider": "jpa"
> > 
> 
> > > },
> > 
> 

> > > "user": {
> > 
> 
> > > "provider": "jpa"
> > 
> 
> > > },
> > 
> 

> > > "userSessionPersister": {
> > 
> 
> > > "provider": "jpa"
> > 
> 
> > > },
> > 
> 

> > > "timer": {
> > 
> 
> > > "provider": "basic"
> > 
> 
> > > },
> > 
> 

> > > "theme": {
> > 
> 
> > > "default": "keycloak",
> > 
> 
> > > "staticMaxAge": 2592000,
> > 
> 
> > > "cacheTemplates": true,
> > 
> 
> > > "cacheThemes": true,
> > 
> 
> > > "folder": {
> > 
> 
> > > "dir": "${jboss.server.config.dir}/themes"
> > 
> 
> > > }
> > 
> 
> > > },
> > 
> 

> > > "scheduled": {
> > 
> 
> > > "interval": 900
> > 
> 
> > > },
> > 
> 

> > > "connectionsHttpClient": {
> > 
> 
> > > "default": {
> > 
> 
> > > "disable-trust-manager": true
> > 
> 
> > > }
> > 
> 
> > > },
> > 
> 

> > > "connectionsJpa": {
> > 
> 
> > > "default": {
> > 
> 
> > > "dataSource": "java:jboss/datasources/PgDskeycloak",
> > 
> 
> > > "databaseSchema": "update"
> > 
> 
> > > }
> > 
> 
> > > },
> > 
> 

> > > "connectionsInfinispan": {
> > 
> 
> > > "default" : {
> > 
> 
> > > "cacheContainer" : "java:jboss/infinispan/Keycloak"
> > 
> 
> > > }
> > 
> 
> > > }
> > 
> 
> > > }
> > 
> 

> > > CHARLES-EDOUARD GAGNAIRE
> > 
> 
> > > SysAdmin
> > 
> 
> > > c.gagnaire at kreactive.com
> > 
> 
> > > p. 06.27.80.28.53
> > 
> 
> > > LYON "Le Capitole"
> > 
> 
> > > 97, cours Gambetta
> > 
> 
> > > 69481 Lyon Cedex 03
> > 
> 

> > > PARIS
> > 
> 
> > > 16, rue de Turbigo
> > 
> 
> > > 75002 Paris
> > 
> 

> > > 
> > 
> 

> > > _______________________________________________
> > 
> 
> > > keycloak-user mailing list
> > 
> 
> > > keycloak-user at lists.jboss.org
> > 
> 
> > > https://lists.jboss.org/mailman/listinfo/keycloak-user
> > 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20151218/3c4759f4/attachment-0001.html 


More information about the keycloak-user mailing list