[keycloak-user] Galera Replication and Caching

[Jared Blashka]

Both of our keycloak nodes are living in the same physical
datacenter+networking space and are the only two nodes in an infinispan
cluster; they're just each using a different Galera DB (and these are
clustered synchronously together along with a 3rd Galera node). We were
trying to validate that DB replication wouldn't break like it did for a
similar configuration we were using earlier (using asynchronous DB
replication). So the DB replication isn't breaking and appears to be
functioning as expected, but it looks like there's data cached by each
Keycloak node that doesn't get refreshed from the DB nor corrected by
infinispan. So far the only thing we've noticed are changes not appearing
in the Admin UI e.g. Realm/Client changes performed on Keycloak01 don't
appear in the UI for Keycloak02 but *do* appear in Galera02. The issue
doesn't seem to extend to client sessions; we haven't heard any issues of
people being asked to log in multiple times.

I'd be happy to run any specific tests in our set up if you want additional
info.

Jared

On Mon, Oct 17, 2016 at 2:34 PM, Stian Thorgersen <sthorger at redhat.com>
wrote:

> Just to point out the maybe not so obvious - all realm configuration
> including clients are cached in an Infinispan invalidation cache. I've got
> no idea how to setup the Infinispan invalidation caches cross data centers,
> but that would be required for entries to be re-loaded in one DC when
> updated in another DC.
>
> On 13 October 2016 at 17:08, Marek Posolda <mposolda at redhat.com> wrote:
>
>> And are also both Keycloak nodes in the same infinispan cluster?
>>
>> Marek
>>
>> Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
>> > We've got synchronous replication enabled. I've looked in the DB
>> > tables for both galera nodes and the data is there. e.g. both DB nodes
>> > have client 'myclient' but the UI for Keycloak node 2 doesn't list a
>> > 'myclient'. But Keycloak will error if you try to add 'myclient'
>> > saying it already exists.
>> >
>> > On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda at redhat.com
>> > <mailto:mposolda at redhat.com>> wrote:
>> >
>> >     Then it's probably related to the Galera cluster rather then to
>> >     caching...
>> >
>> >     Do you have DB configured with synchronous replication (eg.
>> >     inserting some record on DB1 is successfully finished after the
>> >     record is successfully replicated to DB2 too) ?
>> >
>> >     You can maybe compare with the configuration in my docker image
>> >     https://github.com/mposolda/keycloak-mariadb
>> >     <https://github.com/mposolda/keycloak-mariadb> . I can't recall to
>> >     see any issue like this, but not sure about other aspects of my
>> >     configuration (performance etc).
>> >
>> >     Marek
>> >
>> >
>> >     On 12/10/16 19:08, Jared Blashka wrote:
>> >>     We're already running 1.9.8.Final. Our previous configuration was
>> >>     using 2 clustered nodes configured against the same DB node and
>> >>     we didn't run into this issue.
>> >>
>> >>     On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
>> >>     <mposolda at redhat.com <mailto:mposolda at redhat.com>> wrote:
>> >>
>> >>         Which Keycloak version are you using? If it's older than
>> >>         1.9.8.Final,
>> >>         then it's suggested to upgrade as there were caching fixes
>> >>         meanwhile.
>> >>
>> >>         There is also possibility to disable caching in
>> >>         keycloak-server.json (or
>> >>         in standalone.xml in latest version). It's mentioned in the
>> >>         docs how to
>> >>         do it.
>> >>
>> >>         Finally it may also help if you have opportunity to try with
>> >>         2 Keycloak
>> >>         cluster nodes configured against same DB node. This may help
>> >>         to better
>> >>         isolate the problem and see if it's related to caching or to
>> >>         MariaDB
>> >>         cluster.
>> >>
>> >>         Marek
>> >>
>> >>         On 10/10/16 22:31, Josh Cain wrote:
>> >>         > Hi all,
>> >>         >
>> >>         > We're running into a problem with a couple of MariaDB
>> >>         instances +
>> >>         > Galera.  When I go to add a client on the first Keycloak
>> >>         node/DB (we'll
>> >>         > call it DB01), it add successfully.  I can then go to the
>> >>         second
>> >>         > Keycloak Node/DB (call this one DB02) and do not see the
>> >>         client on the
>> >>         > 'clients' list.  However, if I were to add the same client
>> >>         on DB02, I
>> >>         > get the expected 'client with ID already exists' message.
>> >>         What's more,
>> >>         > if I bounce the Keycloak node that talks to DB02, the
>> >>         client list
>> >>         > populates with the new entry added at DB01.
>> >>         >
>> >>         > Was guessing it's some kind of caching issue - is there a
>> >>         setting where
>> >>         > I can alter this behavior?
>> >>         >
>> >>
>> >>         _______________________________________________
>> >>         keycloak-user mailing list
>> >>         keycloak-user at lists.jboss.org
>> >>         <mailto:keycloak-user at lists.jboss.org>
>> >>         https://lists.jboss.org/mailman/listinfo/keycloak-user
>> >>         <https://lists.jboss.org/mailman/listinfo/keycloak-user>
>> >>
>> >>
>> >
>> >
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>