[keycloak-user] Does Keycloak need sticky session at the load balancer?

Thu Aug 23 07:25:20 EDT 2018

Thanks for the reply Sebastian!

Note, that IP Multicasting is disabled in many data centers (I have never
> found out why they do it, but I've seen it many, many times). So make sure
> your cluster forms correctly (just grep logs and look for "view").
>

I thought about that. Then, I used tcpdump, and I can see the multicast
packets from both Keycloak replicas. However, it seems that these packets
are being ignored.

root at Keycloak01:/# tcpdump -i eth0 port 7600 or port 55200 or port 45700 or
> port 45688 or port 23364 or port 4712 or port 4713
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
> 11:13:36.540080 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length 83
> 11:13:41.288449 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length 83
> 11:13:46.342606 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length 83
>

> root at keycloak02:/# tcpdump -i eth0 port 7600 or port 55200 or port 45700
> or port 45688 or port 23364 or port 4712 or port 4713
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
> 11:12:14.218317 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length 83
> 11:12:23.146798 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length 83
> 11:12:27.201888 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length 83
>

Here go the log entries. I filtered by “view”. This is from Keycloak01.

> ^[[0m^[[0m11:16:57,896 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-4) ISPN000094: Received new cluster view for channel ejb:
> [keycloak01|0] (1) [keycloak01]
> ^[[0m^[[0m11:16:57,896 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-2) ISPN000094: Received new cluster view for channel ejb:
> [keycloak01|0] (1) [keycloak01]
> ^[[0m^[[0m11:16:57,897 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
> [keycloak01|0] (1) [keycloak01]
> ^[[0m^[[0m11:16:57,898 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-3) ISPN000094: Received new cluster view for channel ejb:
> [keycloak01|0] (1) [keycloak01]
> ^[[0m^[[0m11:16:57,962 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
> [keycloak01|0] (1) [keycloak01]
>

I expected it to be only one.  I mean, I first started Keycloak01, and just
then Keycloak02. Next, we have the logs from Keycloak02.

^[[0m^[[0m11:17:34,950 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-3) ISPN000094: Received new cluster view for channel ejb:
> [keycloak02|0] (1) [keycloak02]
> ^[[0m^[[0m11:17:34,952 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-4) ISPN000094: Received new cluster view for channel ejb:
> [keycloak02|0] (1) [keycloak02]
> ^[[0m^[[0m11:17:34,957 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
> [keycloak02|0] (1) [keycloak02]
> ^[[0m^[[0m11:17:34,957 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-2) ISPN000094: Received new cluster view for channel ejb:
> [keycloak02|0] (1) [keycloak02]
> ^[[0m^[[0m11:17:35,052 INFO
>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
> [keycloak02|0] (1) [keycloak02
>

They are similar. It seems that both applications are not seeing each
other. At first, I thought that the problem was caused by “owners=1”
configuration (the lack of data synchronization between replicas). I then
changed it to “owners=2”, but still, if I log in the Keycloak01 and then
force my request to go two Keycloak02, my session is not there, and I am
requested to log in again.

Do you need some other log entries or configuration files?

Again, thanks for your reply and help!

On Thu, Aug 23, 2018 at 5:24 AM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

>
>
> On Wed, Aug 22, 2018 at 10:24 PM Rafael Weingärtner <
> rafaelweingartner at gmail.com> wrote:
>
>> Hello Keycloakers,
>>
>> I have some doubts regarding Keycloak and load balancers. I set up two
>> keycloak replicas to provide HA. To start them I am using “./standalone.sh
>> --server-config=standalone-ha.xml”.  I am assuming that they will use
>> multicast to replicate information between nodes, right?
>>
>
> That is correct. It uses PING protocol, which in turn uses IP Multicasting
> for discovery.
>
> Note, that IP Multicasting is disabled in many data centers (I have never
> found out why they do it, but I've seen it many, many times). So make sure
> your cluster forms correctly (just grep logs and look for "view").
>
>
>> Then, I set up a load balancer layer using Apache HTTPD and AJP connector
>> via 8009 port. To make everything work I needed to use sticky session;
>> otherwise, the login would never happen. I am fine with the sticky
>> session,
>> however, if I stop one of the replicas where the user is logged in, when
>> the user access Keycloak again, he/she is asked to present the credentials
>> as if he/she was not logged in the other Keycloak replica. Is that the
>> expected behavior?
>>
>
> My intuition tells me that your cluster didn't form correctly (as I
> mentioned before, grep the logs and look for "view" generated by JGroups).
> Therefore, if you enable sticky session, all your requests get to the same
> Keycloak instance, which has everything in the local cache. That's why it
> works fine.
>
>
>>
>> Is there some troubleshooting or test that I can perform to check if
>> replication is being executed?
>>
>
> Let's start with investigating the logs. Later on we can check JMX.
>
>
>>
>> --
>> Rafael Weingärtner
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
>

-- 
Rafael Weingärtner