[keycloak-user] Not able to setup Keycloak to fully replicate user sessions in cluster

[Jyoti Kumar Singh]

Hi Stian,

Thanks for the reply. I am using below configuration of the
standalone-ha.xml from 3.1.0 version. I just added owners="2" in
"infinispan/Keycloak" for cluster-wide replicas for each cache entry.

#standalone-ha.xml:- attached

Also I am using DC/OS as a container platform, which includes Marathon as a
load balancer (LB) and two container runtimes (Docker and Mesos) for the
deployment on cloud.

I could see below logs are rolling in Node#2(nodeagent16) once
Node#1(nodeagent15) goes down. But when I am bringing Node#1 again, request
is being transferred from LB to Node#1 again and I am not seeing any logs
related to Cache session are rolling in Node#1, hence user's session is not
recognized by Node#1 and he is asked to login again.

Currently I am not very sure whether multicasting is not working or
discovery protocol is having some issue. Your inputs will help me to
understand the issue in a better way.

#Logs:-

2017-06-10 04:41:56,330 WARN
 [org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy]
(transport-thread--p16-t3) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000314:
Cache authorization lost at least half of the stable members, possible
split brain causing data inconsistency. Current members are [nodeagent16],
lost members are [nodeagent15], stable members are [nodeagent15,
nodeagent16]
2017-06-10 04:41:56,332 WARN
 [org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy]
(transport-thread--p16-t3) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000314:
Cache sessions lost at least half of the stable members, possible split
brain causing data inconsistency. Current members are [nodeagent16], lost
members are [nodeagent15], stable members are [nodeagent15, nodeagent16]
2017-06-10 04:41:56,333 WARN
 [org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy]
(transport-thread--p16-t3) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000314:
Cache work lost at least half of the stable members, possible split brain
causing data inconsistency. Current members are [nodeagent16], lost members
are [nodeagent15], stable members are [nodeagent16, nodeagent15]
2017-06-10 04:41:56,334 WARN
 [org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy]
(transport-thread--p16-t3) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000314:
Cache offlineSessions lost at least half of the stable members, possible
split brain causing data inconsistency. Current members are [nodeagent16],
lost members are [nodeagent15], stable members are [nodeagent15,
nodeagent16]
2017-06-10 04:41:56,336 WARN
 [org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy]
(transport-thread--p16-t3) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000314:
Cache loginFailures lost at least half of the stable members, possible
split brain causing data inconsistency. Current members are [nodeagent16],
lost members are [nodeagent15], stable members are [nodeagent15,
nodeagent16]
2017-06-10 04:41:56,509 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-2,ee,nodeagent16) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000094:
Received new cluster view for channel web: [nodeagent16|10] (1)
[nodeagent16]
2017-06-10 04:41:56,512 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-2,ee,nodeagent16) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000094:
Received new cluster view for channel ejb: [nodeagent16|10] (1)
[nodeagent16]
2017-06-10 04:41:56,513 INFO
 [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
(Incoming-2,ee,nodeagent16) [nodeagent16] KEYCLOAK 3.1.0-0.1 ISPN000094:
Received new cluster view for channel hibernate: [nodeagent16|10] (1)
[nodeagent16]



On Fri, Jun 9, 2017 at 10:28 AM, Stian Thorgersen <sthorger at redhat.com>
wrote:

> Your configuration is not correct and seems to be from an older version of
> Keycloak. Please take a look at default standalone-ha.xml from 3.1 for the
> correct cache configs.
>
> You also need to get cluster communication working properly. Make sure the
> nodes see each other. When you start new nodes something should happen in
> the log in other nodes. In a cloud environment this can be tricky (you
> haven't said which one) as multicasting usually doesn't work and you need
> to use a different discovery protocol.
>
> On 7 June 2017 at 16:17, Jyoti Kumar Singh <jyoti.tech90 at gmail.com> wrote:
>
>> Hi Team,
>>
>> We are setting up keycloak:3.1.0.Final in a cluster mode for HA with full
>> user sessions replication in a cloud system, i.e. when one node goes down
>> then user will keep logged in on other node.
>>
>> I have setup cluster by using standalone-ha.xml and having infinispan
>> cache
>> as mentioned below:-
>>
>> <cache-container name="keycloak" jndi-name="infinispan/Keycloak">
>>                 <transport lock-timeout="60000"/>
>>                 <invalidation-cache name="realms" mode="SYNC"/>
>>                 <invalidation-cache name="users" mode="SYNC"/>
>>                 <distributed-cache name="sessions" mode="SYNC"
>> owners="2"/>
>>                 <distributed-cache name="loginFailures" mode="SYNC"
>> owners="2"/>
>> </cache-container>
>>
>> Every thing works fine except below use case:-
>>
>> 1. Node 1 and Node 2 both are up and user logged in - User session is
>> getting generated by Node 1
>> 2. Node 1 is now stopped and user session is getting replicated in Node 2
>> -
>> User is still able to use the Keycloak console
>> 3. Node 1 is up again and request is being transferred from LB to Node 1 -
>> User is asked to log in again because session cache is not replicated to
>>     Node 1 immediately once it is up
>>
>> I saw one option to add *start="EAGER" *in cache-container to fix this but
>> looks like with latest version of WildFly it is no longer supported. Do we
>> have any other way to fix this issue ?
>>
>>
>> --
>>
>> *With Regards, Jyoti Kumar Singh*
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>


-- 

*With Regards, Jyoti Kumar Singh*