[keycloak-user] Does Keycloak need sticky session at the load balancer?
Rafael Weingärtner
rafaelweingartner at gmail.com
Wed Aug 29 08:42:42 EDT 2018
Thanks!
The problem was caused by firewalld blocking Multicast traffic.
On Fri, Aug 24, 2018 at 7:28 AM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:
> Great write-up! Bookmarked!
>
> On Thu, Aug 23, 2018 at 4:36 PM Bela Ban <bban at redhat.com> wrote:
>
>> Have you checked
>> https://github.com/belaban/workshop/blob/master/slides/
>> admin.adoc#problem-1-members-don-t-find-each-other?
>>
>> On 23/08/18 13:53, Sebastian Laskawiec wrote:
>> > +Bela Ban <mailto:bban at redhat.com>
>> >
>> > As I expected, the cluster doesn't form.
>> >
>> > I'm not sure where and why those UDP discovery packets are rejected. I
>> > just stumbled upon this thread [1], which you may find useful. Maybe
>> > Bela will also have an idea what's going on there.
>> >
>> > If you won't manage to get UDP working, you can always fall back into
>> > TCP (and MPING).
>> >
>> > [1] https://serverfault.com/questions/211482/tools-to-
>> test-multicast-routing
>> >
>> > On Thu, Aug 23, 2018 at 1:26 PM Rafael Weingärtner
>> > <rafaelweingartner at gmail.com <mailto:rafaelweingartner at gmail.com>>
>> wrote:
>> >
>> > Thanks for the reply Sebastian!
>> >
>> >
>> > Note, that IP Multicasting is disabled in many data centers (I
>> > have never found out why they do it, but I've seen it many, many
>> > times). So make sure your cluster forms correctly (just grep
>> > logs and look for "view").
>> >
>> >
>> > I thought about that. Then, I used tcpdump, and I can see the
>> > multicast packets from both Keycloak replicas. However, it seems
>> > that these packets are being ignored.
>> >
>> > root at Keycloak01:/# tcpdump -i eth0 port 7600 or port 55200 or
>> > port 45700 or port 45688 or port 23364 or port 4712 or port 4713
>> > tcpdump: verbose output suppressed, use -v or -vv for full
>> > protocol decode
>> > listening on eth0, link-type EN10MB (Ethernet), capture size
>> > 262144 bytes
>> > 11:13:36.540080 IP keycloak02.local.55200 > 230.0.0.4.45688:
>> > UDP, length 83
>> > 11:13:41.288449 IP keycloak02.local.55200 > 230.0.0.4.45688:
>> > UDP, length 83
>> > 11:13:46.342606 IP keycloak02.local.55200 > 230.0.0.4.45688:
>> > UDP, length 83
>> >
>> >
>> > root at keycloak02:/# tcpdump -i eth0 port 7600 or port 55200 or
>> > port 45700 or port 45688 or port 23364 or port 4712 or port 4713
>> > tcpdump: verbose output suppressed, use -v or -vv for full
>> > protocol decode
>> > listening on eth0, link-type EN10MB (Ethernet), capture size
>> > 262144 bytes
>> > 11:12:14.218317 IP Keycloak01.local.55200 > 230.0.0.4.45688:
>> > UDP, length 83
>> > 11:12:23.146798 IP Keycloak01.local.55200 > 230.0.0.4.45688:
>> > UDP, length 83
>> > 11:12:27.201888 IP Keycloak01.local.55200 > 230.0.0.4.45688:
>> > UDP, length 83
>> >
>> >
>> >
>> > Here go the log entries. I filtered by “view”. This is from
>> Keycloak01.
>> >
>> > ^[[0m^[[0m11:16:57,896 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-4) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak01|0] (1) [keycloak01]
>> > ^[[0m^[[0m11:16:57,896 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-2) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak01|0] (1) [keycloak01]
>> > ^[[0m^[[0m11:16:57,897 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-1) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak01|0] (1) [keycloak01]
>> > ^[[0m^[[0m11:16:57,898 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-3) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak01|0] (1) [keycloak01]
>> > ^[[0m^[[0m11:16:57,962 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-1) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak01|0] (1) [keycloak01]
>> >
>> >
>> > I expected it to be only one. I mean, I first started Keycloak01,
>> > and just then Keycloak02. Next, we have the logs from Keycloak02.
>> >
>> > ^[[0m^[[0m11:17:34,950 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-3) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak02|0] (1) [keycloak02]
>> > ^[[0m^[[0m11:17:34,952 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-4) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak02|0] (1) [keycloak02]
>> > ^[[0m^[[0m11:17:34,957 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-1) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak02|0] (1) [keycloak02]
>> > ^[[0m^[[0m11:17:34,957 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-2) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak02|0] (1) [keycloak02]
>> > ^[[0m^[[0m11:17:35,052 INFO
>> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
>> > (MSC service thread 1-1) ISPN000094: Received new cluster view
>> > for channel ejb: [keycloak02|0] (1) [keycloak02
>> >
>> >
>> > They are similar. It seems that both applications are not seeing
>> > each other. At first, I thought that the problem was caused by
>> > “owners=1” configuration (the lack of data synchronization between
>> > replicas). I then changed it to “owners=2”, but still, if I log in
>> > the Keycloak01 and then force my request to go two Keycloak02, my
>> > session is not there, and I am requested to log in again.
>> >
>> > Do you need some other log entries or configuration files?
>> >
>> > Again, thanks for your reply and help!
>> >
>> > On Thu, Aug 23, 2018 at 5:24 AM, Sebastian Laskawiec
>> > <slaskawi at redhat.com <mailto:slaskawi at redhat.com>> wrote:
>> >
>> >
>> >
>> > On Wed, Aug 22, 2018 at 10:24 PM Rafael Weingärtner
>> > <rafaelweingartner at gmail.com
>> > <mailto:rafaelweingartner at gmail.com>> wrote:
>> >
>> > Hello Keycloakers,
>> >
>> > I have some doubts regarding Keycloak and load balancers. I
>> > set up two
>> > keycloak replicas to provide HA. To start them I am using
>> > “./standalone.sh
>> > --server-config=standalone-ha.xml”. I am assuming that
>> they
>> > will use
>> > multicast to replicate information between nodes, right?
>> >
>> >
>> > That is correct. It uses PING protocol, which in turn uses IP
>> > Multicasting for discovery.
>> >
>> > Note, that IP Multicasting is disabled in many data centers (I
>> > have never found out why they do it, but I've seen it many, many
>> > times). So make sure your cluster forms correctly (just grep
>> > logs and look for "view").
>> >
>> > Then, I set up a load balancer layer using Apache HTTPD and
>> > AJP connector
>> > via 8009 port. To make everything work I needed to use
>> > sticky session;
>> > otherwise, the login would never happen. I am fine with the
>> > sticky session,
>> > however, if I stop one of the replicas where the user is
>> > logged in, when
>> > the user access Keycloak again, he/she is asked to present
>> > the credentials
>> > as if he/she was not logged in the other Keycloak replica.
>> > Is that the
>> > expected behavior?
>> >
>> >
>> > My intuition tells me that your cluster didn't form correctly
>> > (as I mentioned before, grep the logs and look for "view"
>> > generated by JGroups). Therefore, if you enable sticky session,
>> > all your requests get to the same Keycloak instance, which has
>> > everything in the local cache. That's why it works fine.
>> >
>> >
>> > Is there some troubleshooting or test that I can perform to
>> > check if
>> > replication is being executed?
>> >
>> >
>> > Let's start with investigating the logs. Later on we can check
>> JMX.
>> >
>> >
>> > --
>> > Rafael Weingärtner
>> > _______________________________________________
>> > keycloak-user mailing list
>> > keycloak-user at lists.jboss.org
>> > <mailto:keycloak-user at lists.jboss.org>
>> > https://lists.jboss.org/mailman/listinfo/keycloak-user
>> >
>> >
>> >
>> >
>> > --
>> > Rafael Weingärtner
>> >
>>
>> --
>> Bela Ban | http://www.jgroups.org
>>
>>
--
Rafael Weingärtner
More information about the keycloak-user
mailing list