Re: [keycloak-dev] Questions about Standalone HA

Tonnis, Some comments in line. On Fri, Sep 1, 2017 at 4:10 PM Tonnis Wildeboer <tonnis(a)autonomic.ai <mailto:tonnis@autonomic.ai>> wrote: Hello, I posted similar questions on keycloak-user, but it doesn't seem to be the right audience. Please redirect me if these are not appropriate for this group. *Background:* I am running Keycloak in a kubernetes cluster with a shared postgres (RDS) db. Everything is hosted on AWS. The Keycloak instances are deployed using Helm. I have read the clustering and caching documentation and from that it seems that the appropriate clustering mode in this scenario would be "Standalone Clustered Mode". Therefore, I am using the "jboss/keycloak-ha-postgres" Docker image. Since I am using the nginx Ingress controller I have the prescribedPROXY_ADDRESS_FORWARDING=true environment variable. Upon inspection of the Docker image, however, I noticed that the $JBOSS_HOME/standalone/configuration/standalone-ha.xml file in that image does not have the proxy-address-forwarding="${env.PROXY_ADDRESS_FORWARDING}" attribute in the <http-listener ...> element. I also noticed that the jboss-dockerfiles/keycloak-server base image has a sed command to add this to the standalone.xml file but not to the standalone-ha.xml file. (Also, for the benefit of others interested in this configuration, I have configured session affinity in the Ingress to avoid the default round-robin routing, which causes infinite redirects to Keycloak, bouncing between instances.) This is probably your first sign of an issue. This indicates that your nodes aren't talking to one another, session affinity is not actually required (I have an AWS deployment today that works fine) Of the examples I have found via Google searches, I have not found examples of deploying Keycloak this way, which is surprising. I have seen examples with a single instance using the standalone postgres image, but not standalone-ha ("Standalone Clustered"). *So here are my questions:* 1. Why doesn't the base jboss-dockerfiles/keycloak-server image also modify the standalone-ha.xml file too, in the same way it modifies the standalone.xml file: (https://github.com/jboss-dockerfiles/keycloak/blob/0a54ccaccd5e27e75105b9... Probably because the docker use case has only considered simple development. For what its worth, I don't deploy any prebuilt docker containers, everything I deploy is a customized image in some way. 2. I assume the discovery of the nodes between one another (for the sake of the shared caches) is accomplished through multicast as long as they are on the same subnet. Correct? Normally, yes. However, in AWS multicast doesn't work (even when in the same VPC/subnet). To work around this, you can leverage JDBC_PING in jgroups. https://developer.jboss.org/wiki/JDBCPING 3. What should I be looking for in the logs to see whether the peer node discovery is happening? What logging level do I need to see this? You'll see log messages from jgroups indicating that the cluster is alive and a peer was discovered. I turned this up very high to give more information. 4. If the nodes (pods) are in different AWS VPCs and there is no explicit routing set up between them such that they cannot discover one another, but they do share the same postgres instance, is there any harm in this? I would assume that each node would take responsibility for being the cache owner, and that is not a problem. If your cache isn't replicating, users will lose sessions frequently. 5. Is there any other documentation, etc that I should be looking at? The link I put above, and probably the existing clustering docs which you may have already reviewed. Thank you, --Tonnis _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/keycloak-dev