[keycloak-user] Expose JGroups ports in Docker keycloak-ha-postgres

[Iván Perdomo]

Hi,

I'm also interesting on running Keycloak HA with Kubernetes/Google Cloud
Platform.

> JGroups members use multicast communication over UDP to broadcast
> their presence to other instances on a network. Google Cloud
> Platform, like most cloud providers and enterprise networks, does not
> support multicast

https://cloudplatform.googleblog.com/2016/02/JGroups-based-clustering-and-node-discovery-with-Google-Cloud-Storage.html

Do we need to take the route and use GOOGLE_PING method for node discovery?

Any hints on this topic are quite valuable.

Thanks,

On 11/09/2016 02:54 PM, Staffan wrote:
> I have verified that `hostname -i` works with Minikube, but not yet a
> multi-node cluster.
> 
> Created PR https://github.com/jboss-dockerfiles/keycloak/pull/59 for the
> official HA docker image.
> 
> Is the following warning in keycloak logs something that affects clustering?
> 
> WARN  [org.jboss.as.txn] (ServerService Thread Pool -- 49) WFLYTX0013: Node
> identifier property is set to the default value. Please make sure it is
> unique.
> 
> It can be remedied using
> https://github.com/Reposoft/keycloak-ha-kubernetes/commit/413665f0c0827f8fa35379cc1f78098124290cd8
> but I have avoided config file changes.
> 
> /Staffan
> 
> On Tue, Nov 8, 2016 at 3:06 PM, Alan Gibson <alan.gibson at gmail.com> wrote:
> 
>> Hi Staffan,
>>
>> We've got 3 clustered Keycloak nodes running in Docker with host (not
>> bridge) networking and managed by Mesos/Marathon. Cluster communications
>> run over UDP. We start them with the following command.
>>
>> /opt/jboss/docker-entrypoint.sh -Dkeycloak.migration.action={{keycloak_migration_action}}
>> -Dkeycloak.migration.provider={{keycloak_migration_provider}}
>> -Dkeycloak.migration.file={{keycloak_migration_file}}
>> -Dkeycloak.migration.strategy={{keycloak_migration_strategy}}
>> -Djboss.jgroups.stack={{keycloak_jgroups_stack}}
>> -Djboss.jgroups.udp.port={{keycloak_jgroups_udp_port}}
>> -Djboss.jgroups.udp.multicast.port={{keycloak_jgroups_udp_multicast_port}}
>> -Djboss.jgroups.udp.fd.port={{keycloak_jgroups_udp_fd_port}}
>> -Djboss.management.http.port=$PORT1 -Djboss.http.port=$PORT0
>> -Djboss.bind.address.private=$(hostname -i) -b 0.0.0.0 -bmanagement
>> 0.0.0.0 --server-config standalone-ha.xml
>>
>> keycloak_jgroups_stack: udp
>> keycloak_jgroups_udp_port: 5520
>> keycloak_jgroups_udp_multicast_port: 4568
>> keycloak_jgroups_udp_fd_port: 5420
>>
>> The magic ingredient is using getting the jboss.bind.address.private
>> address from the shell with $(hostname -i). Note that this is definitely
>> not foolproof, so YMMV.
>>
>> Br, Alan
>>
>> On Tue, Nov 8, 2016 at 11:59 AM, Staffan <solsson at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I've tried in different docker environments (compose, kubernetes,
>>> standalone) to get a HA setup running using https://hub.docker.com/r/
>>> jboss/keycloak-ha-postgres/
>>> <https://hub.docker.com/r/jboss/keycloak-ha-postgres/>.
>>>
>>> Keycloak nodes start, but are unaware of each other. I fail to reach the
>>> JGroups ports from any other container or host system. That is expected,
>>> as
>>> https://keycloak.gitbooks.io/server-installation-and-configu
>>> ration/content/v/2.3/topics/clustering/multicast.html
>>> advises you to configure jboss.bind.address.private.
>>>
>>> But when I try -Djboss.bind.address.private=0.0.0.0 there's an error
>>> during
>>> startup:
>>>
>>> MSC000001: Failed to start service jboss.jgroups.channel.ee:
>>> org.jboss.msc.service.StartException in service jboss.jgroups.channel.ee:
>>> java.security.PrivilegedActionException: java.net.BindException: [UDP] /
>>> 0.0.0.0 is not a valid address on any local network interface
>>>     at org.wildfly.clustering.jgroups.spi.service.ChannelBuilder.start(
>>> ChannelBuilder.java:80)
>>> Caused by: java.security.PrivilegedActionException:
>>> java.net.BindException:
>>> [UDP] /0.0.0.0 is not a valid address on any local network interface
>>>     at org.wildfly.security.manager.WildFlySecurityManager.doChecked(
>>> WildFlySecurityManager.java:640)
>>> Caused by: java.net.BindException: [UDP] /0.0.0.0 is not a valid address
>>> on
>>> any local network interface
>>>     at org.jgroups.util.Util.checkIfValidAddress(Util.java:3522)
>>>
>>> ... or if I switch to stack="tcp" in the jgroups subsystem:
>>>
>>> MSC000001: Failed to start service jboss.jgroups.channel.ee:
>>> org.jboss.msc.service.StartException in service jboss.jgroups.channel.ee:
>>> java.security.PrivilegedActionException: java.net.BindException: [TCP] /
>>> 0.0.0.0 is not a valid address on any local network interface
>>>
>>> I guess this is a generic Wildfly topic, but I'm curious how the official
>>> Keycloak docker containers are tested. In a docker environment, what can
>>> we
>>> bind to other than 0.0.0.0 or 127.0.0.1? Is there a way to allow a
>>> "privileged action"?
>>>
>>> regards
>>> Staffan Olsson
>>> _______________________________________________
>>> keycloak-user mailing list
>>> keycloak-user at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>>
>>
>>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
> 

-- 
Iván