[keycloak-user] Does Keycloak need sticky session at the load balancer?

Thu Aug 23 07:53:23 EDT 2018

+Bela Ban <bban at redhat.com>

As I expected, the cluster doesn't form.

I'm not sure where and why those UDP discovery packets are rejected. I just
stumbled upon this thread [1], which you may find useful. Maybe Bela will
also have an idea what's going on there.

If you won't manage to get UDP working, you can always fall back into TCP
(and MPING).

[1] https://serverfault.com/questions/211482/tools-to-test-multicast-routing

On Thu, Aug 23, 2018 at 1:26 PM Rafael Weingärtner <
rafaelweingartner at gmail.com> wrote:

> Thanks for the reply Sebastian!
>
>
> Note, that IP Multicasting is disabled in many data centers (I have never
>> found out why they do it, but I've seen it many, many times). So make sure
>> your cluster forms correctly (just grep logs and look for "view").
>>
>
> I thought about that. Then, I used tcpdump, and I can see the multicast
> packets from both Keycloak replicas. However, it seems that these packets
> are being ignored.
>
> root at Keycloak01:/# tcpdump -i eth0 port 7600 or port 55200 or port 45700
>> or port 45688 or port 23364 or port 4712 or port 4713
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
>> 11:13:36.540080 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length
>> 83
>> 11:13:41.288449 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length
>> 83
>> 11:13:46.342606 IP keycloak02.local.55200 > 230.0.0.4.45688: UDP, length
>> 83
>>
>
>
>> root at keycloak02:/# tcpdump -i eth0 port 7600 or port 55200 or port 45700
>> or port 45688 or port 23364 or port 4712 or port 4713
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
>> 11:12:14.218317 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length
>> 83
>> 11:12:23.146798 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length
>> 83
>> 11:12:27.201888 IP Keycloak01.local.55200 > 230.0.0.4.45688: UDP, length
>> 83
>>
>
>
> Here go the log entries. I filtered by “view”. This is from Keycloak01.
>
>> ^[[0m^[[0m11:16:57,896 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-4) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak01|0] (1) [keycloak01]
>> ^[[0m^[[0m11:16:57,896 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-2) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak01|0] (1) [keycloak01]
>> ^[[0m^[[0m11:16:57,897 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak01|0] (1) [keycloak01]
>> ^[[0m^[[0m11:16:57,898 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-3) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak01|0] (1) [keycloak01]
>> ^[[0m^[[0m11:16:57,962 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak01|0] (1) [keycloak01]
>>
>
> I expected it to be only one.  I mean, I first started Keycloak01, and
> just then Keycloak02. Next, we have the logs from Keycloak02.
>
> ^[[0m^[[0m11:17:34,950 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-3) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak02|0] (1) [keycloak02]
>> ^[[0m^[[0m11:17:34,952 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-4) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak02|0] (1) [keycloak02]
>> ^[[0m^[[0m11:17:34,957 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak02|0] (1) [keycloak02]
>> ^[[0m^[[0m11:17:34,957 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-2) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak02|0] (1) [keycloak02]
>> ^[[0m^[[0m11:17:35,052 INFO
>>  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
>> thread 1-1) ISPN000094: Received new cluster view for channel ejb:
>> [keycloak02|0] (1) [keycloak02
>>
>
> They are similar. It seems that both applications are not seeing each
> other. At first, I thought that the problem was caused by “owners=1”
> configuration (the lack of data synchronization between replicas). I then
> changed it to “owners=2”, but still, if I log in the Keycloak01 and then
> force my request to go two Keycloak02, my session is not there, and I am
> requested to log in again.
>
> Do you need some other log entries or configuration files?
>
> Again, thanks for your reply and help!
>
> On Thu, Aug 23, 2018 at 5:24 AM, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
>
>>
>>
>> On Wed, Aug 22, 2018 at 10:24 PM Rafael Weingärtner <
>> rafaelweingartner at gmail.com> wrote:
>>
>>> Hello Keycloakers,
>>>
>>> I have some doubts regarding Keycloak and load balancers. I set up two
>>> keycloak replicas to provide HA. To start them I am using
>>> “./standalone.sh
>>> --server-config=standalone-ha.xml”.  I am assuming that they will use
>>> multicast to replicate information between nodes, right?
>>>
>>
>> That is correct. It uses PING protocol, which in turn uses IP
>> Multicasting for discovery.
>>
>> Note, that IP Multicasting is disabled in many data centers (I have never
>> found out why they do it, but I've seen it many, many times). So make sure
>> your cluster forms correctly (just grep logs and look for "view").
>>
>>
>>> Then, I set up a load balancer layer using Apache HTTPD and AJP connector
>>> via 8009 port. To make everything work I needed to use sticky session;
>>> otherwise, the login would never happen. I am fine with the sticky
>>> session,
>>> however, if I stop one of the replicas where the user is logged in, when
>>> the user access Keycloak again, he/she is asked to present the
>>> credentials
>>> as if he/she was not logged in the other Keycloak replica. Is that the
>>> expected behavior?
>>>
>>
>> My intuition tells me that your cluster didn't form correctly (as I
>> mentioned before, grep the logs and look for "view" generated by JGroups).
>> Therefore, if you enable sticky session, all your requests get to the same
>> Keycloak instance, which has everything in the local cache. That's why it
>> works fine.
>>
>>
>>>
>>> Is there some troubleshooting or test that I can perform to check if
>>> replication is being executed?
>>>
>>
>> Let's start with investigating the logs. Later on we can check JMX.
>>
>>
>>>
>>> --
>>> Rafael Weingärtner
>>> _______________________________________________
>>> keycloak-user mailing list
>>> keycloak-user at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>>
>
>
> --
> Rafael Weingärtner
>