Hello Keycloakers, I have some doubts regarding Keycloak and load balancers. I set up two keycloak replicas to provide HA. To start them I am using “./standalone.sh --server-config=standalone-ha.xml”. I am assuming that they will use multicast to replicate information between nodes, right?

Then, I set up a load balancer layer using Apache HTTPD and AJP connector via 8009 port. To make everything work I needed to use sticky session; otherwise, the login would never happen. I am fine with the sticky session, however, if I stop one of the replicas where the user is logged in, when the user access Keycloak again, he/she is asked to present the credentials as if he/she was not logged in the other Keycloak replica. Is that the expected behavior?

Is there some troubleshooting or test that I can perform to check if replication is being executed?

-- Rafael Weingärtner _______________________________________________ keycloak-user mailing list keycloak-user(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-user

Thanks for the reply Sebastian! Note, that IP Multicasting is disabled in many data centers (I have never > found out why they do it, but I've seen it many, many times). So make sure > your cluster forms correctly (just grep logs and look for "view"). > I thought about that. Then, I used tcpdump, and I can see the multicast packets from both Keycloak replicas. However, it seems that these packets are being ignored. root@Keycloak01:/# tcpdump -i eth0 port 7600 or port 55200 or port 45700 > or port 45688 or port 23364 or port 4712 or port 4713 > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes > 11:13:36.540080 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length > 83 > 11:13:41.288449 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length > 83 > 11:13:46.342606 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length > 83 > > root@keycloak02:/# tcpdump -i eth0 port 7600 or port 55200 or port 45700 > or port 45688 or port 23364 or port 4712 or port 4713 > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes > 11:12:14.218317 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length > 83 > 11:12:23.146798 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length > 83 > 11:12:27.201888 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length > 83 > Here go the log entries. I filtered by “view”. This is from Keycloak01. > ^[[0m^[[0m11:16:57,896 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-4) ISPN000094: Received new cluster view for channel ejb: > [keycloak01|0] (1) [keycloak01] > ^[[0m^[[0m11:16:57,896 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-2) ISPN000094: Received new cluster view for channel ejb: > [keycloak01|0] (1) [keycloak01] > ^[[0m^[[0m11:16:57,897 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-1) ISPN000094: Received new cluster view for channel ejb: > [keycloak01|0] (1) [keycloak01] > ^[[0m^[[0m11:16:57,898 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-3) ISPN000094: Received new cluster view for channel ejb: > [keycloak01|0] (1) [keycloak01] > ^[[0m^[[0m11:16:57,962 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-1) ISPN000094: Received new cluster view for channel ejb: > [keycloak01|0] (1) [keycloak01] > I expected it to be only one. I mean, I first started Keycloak01, and just then Keycloak02. Next, we have the logs from Keycloak02. ^[[0m^[[0m11:17:34,950 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-3) ISPN000094: Received new cluster view for channel ejb: > [keycloak02|0] (1) [keycloak02] > ^[[0m^[[0m11:17:34,952 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-4) ISPN000094: Received new cluster view for channel ejb: > [keycloak02|0] (1) [keycloak02] > ^[[0m^[[0m11:17:34,957 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-1) ISPN000094: Received new cluster view for channel ejb: > [keycloak02|0] (1) [keycloak02] > ^[[0m^[[0m11:17:34,957 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-2) ISPN000094: Received new cluster view for channel ejb: > [keycloak02|0] (1) [keycloak02] > ^[[0m^[[0m11:17:35,052 INFO > [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service > thread 1-1) ISPN000094: Received new cluster view for channel ejb: > [keycloak02|0] (1) [keycloak02 > They are similar. It seems that both applications are not seeing each other. At first, I thought that the problem was caused by “owners=1” configuration (the lack of data synchronization between replicas). I then changed it to “owners=2”, but still, if I log in the Keycloak01 and then force my request to go two Keycloak02, my session is not there, and I am requested to log in again. Do you need some other log entries or configuration files? Again, thanks for your reply and help! On Thu, Aug 23, 2018 at 5:24 AM, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; wrote: > > > On Wed, Aug 22, 2018 at 10:24 PM Rafael Weingärtner < > rafaelweingartner(a)gmail.com&gt; wrote: > >> Hello Keycloakers, >> >> I have some doubts regarding Keycloak and load balancers. I set up two >> keycloak replicas to provide HA. To start them I am using >> “./standalone.sh >> --server-config=standalone-ha.xml”. I am assuming that they will use >> multicast to replicate information between nodes, right? >> > > That is correct. It uses PING protocol, which in turn uses IP > Multicasting for discovery. > > Note, that IP Multicasting is disabled in many data centers (I have never > found out why they do it, but I've seen it many, many times). So make sure > your cluster forms correctly (just grep logs and look for "view"). > > >> Then, I set up a load balancer layer using Apache HTTPD and AJP connector >> via 8009 port. To make everything work I needed to use sticky session; >> otherwise, the login would never happen. I am fine with the sticky >> session, >> however, if I stop one of the replicas where the user is logged in, when >> the user access Keycloak again, he/she is asked to present the >> credentials >> as if he/she was not logged in the other Keycloak replica. Is that the >> expected behavior? >> > > My intuition tells me that your cluster didn't form correctly (as I > mentioned before, grep the logs and look for "view" generated by JGroups). > Therefore, if you enable sticky session, all your requests get to the same > Keycloak instance, which has everything in the local cache. That's why it > works fine. > > >> >> Is there some troubleshooting or test that I can perform to check if >> replication is being executed? >> > > Let's start with investigating the logs. Later on we can check JMX. > > >> >> -- >> Rafael Weingärtner >> _______________________________________________ >> keycloak-user mailing list >> keycloak-user(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-user > > -- Rafael Weingärtner