3:31 p.m.
Hi all,
We're running into a problem with a couple of MariaDB instances +
Galera. When I go to add a client on the first Keycloak node/DB (we'll
call it DB01), it add successfully. I can then go to the second
Keycloak Node/DB (call this one DB02) and do not see the client on the
'clients' list. However, if I were to add the same client on DB02, I
get the expected 'client with ID already exists' message. What's more,
if I bounce the Keycloak node that talks to DB02, the client list
populates with the new entry added at DB01.
Was guessing it's some kind of caching issue - is there a setting where
I can alter this behavior?
--
Josh Cain | Software Applications Engineer
Identity and Access
Management
Red Hat
+1 256-452-0150
1:45 a.m.
Which Keycloak version are you using? If it's older than 1.9.8.Final,
then it's suggested to upgrade as there were caching fixes meanwhile.
There is also possibility to disable caching in keycloak-server.json (or
in standalone.xml in latest version). It's mentioned in the docs how to
do it.
Finally it may also help if you have opportunity to try with 2 Keycloak
cluster nodes configured against same DB node. This may help to better
isolate the problem and see if it's related to caching or to MariaDB
cluster.
Marek
On 10/10/16 22:31, Josh Cain wrote:
Hi all,
We're running into a problem with a couple of MariaDB instances +
Galera. When I go to add a client on the first Keycloak node/DB (we'll
call it DB01), it add successfully. I can then go to the second
Keycloak Node/DB (call this one DB02) and do not see the client on the
'clients' list. However, if I were to add the same client on DB02, I
get the expected 'client with ID already exists' message. What's more,
if I bounce the Keycloak node that talks to DB02, the client list
populates with the new entry added at DB01.
Was guessing it's some kind of caching issue - is there a setting where
I can alter this behavior?
12:08 p.m.
We're already running 1.9.8.Final. Our previous configuration was using 2
clustered nodes configured against the same DB node and we didn't run into
this issue.
On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda <mposolda(a)redhat.com> wrote:
Which Keycloak version are you using? If it's older than
1.9.8.Final,
then it's suggested to upgrade as there were caching fixes meanwhile.
There is also possibility to disable caching in keycloak-server.json (or
in standalone.xml in latest version). It's mentioned in the docs how to
do it.
Finally it may also help if you have opportunity to try with 2 Keycloak
cluster nodes configured against same DB node. This may help to better
isolate the problem and see if it's related to caching or to MariaDB
cluster.
Marek
On 10/10/16 22:31, Josh Cain wrote:
> Hi all,
>
> We're running into a problem with a couple of MariaDB instances +
> Galera. When I go to add a client on the first Keycloak node/DB (we'll
> call it DB01), it add successfully. I can then go to the second
> Keycloak Node/DB (call this one DB02) and do not see the client on the
> 'clients' list. However, if I were to add the same client on DB02, I
> get the expected 'client with ID already exists' message. What's more,
> if I bounce the Keycloak node that talks to DB02, the client list
> populates with the new entry added at DB01.
>
> Was guessing it's some kind of caching issue - is there a setting where
> I can alter this behavior?
>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user
3:42 p.m.
Then it's probably related to the Galera cluster rather then to caching...
Do you have DB configured with synchronous replication (eg. inserting
some record on DB1 is successfully finished after the record is
successfully replicated to DB2 too) ?
You can maybe compare with the configuration in my docker image
https://github.com/mposolda/keycloak-mariadb . I can't recall to see any
issue like this, but not sure about other aspects of my configuration
(performance etc).
Marek
On 12/10/16 19:08, Jared Blashka wrote:
We're already running 1.9.8.Final. Our previous configuration was
using 2 clustered nodes configured against the same DB node and we
didn't run into this issue.
On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
Which Keycloak version are you using? If it's older than 1.9.8.Final,
then it's suggested to upgrade as there were caching fixes meanwhile.
There is also possibility to disable caching in
keycloak-server.json (or
in standalone.xml in latest version). It's mentioned in the docs
how to
do it.
Finally it may also help if you have opportunity to try with 2
Keycloak
cluster nodes configured against same DB node. This may help to better
isolate the problem and see if it's related to caching or to MariaDB
cluster.
Marek
On 10/10/16 22:31, Josh Cain wrote:
> Hi all,
>
> We're running into a problem with a couple of MariaDB instances +
> Galera. When I go to add a client on the first Keycloak node/DB
(we'll
> call it DB01), it add successfully. I can then go to the second
> Keycloak Node/DB (call this one DB02) and do not see the client
on the
> 'clients' list. However, if I were to add the same client on
DB02, I
> get the expected 'client with ID already exists' message.
What's more,
> if I bounce the Keycloak node that talks to DB02, the client list
> populates with the new entry added at DB01.
>
> Was guessing it's some kind of caching issue - is there a
setting where
> I can alter this behavior?
>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org <mailto:keycloak-user@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-user
<https://lists.jboss.org/mailman/listinfo/keycloak-user>
4:27 p.m.
We've got synchronous replication enabled. I've looked in the DB tables for
both galera nodes and the data is there. e.g. both DB nodes have client
'myclient' but the UI for Keycloak node 2 doesn't list a 'myclient'.
But
Keycloak will error if you try to add 'myclient' saying it already exists.
On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda(a)redhat.com> wrote:
Then it's probably related to the Galera cluster rather then to
caching...
Do you have DB configured with synchronous replication (eg. inserting some
record on DB1 is successfully finished after the record is successfully
replicated to DB2 too) ?
You can maybe compare with the configuration in my docker image
https://github.com/mposolda/keycloak-mariadb . I can't recall to see any
issue like this, but not sure about other aspects of my configuration
(performance etc).
Marek
On 12/10/16 19:08, Jared Blashka wrote:
We're already running 1.9.8.Final. Our previous configuration was using 2
clustered nodes configured against the same DB node and we didn't run into
this issue.
On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda <mposolda(a)redhat.com>
wrote:
> Which Keycloak version are you using? If it's older than 1.9.8.Final,
> then it's suggested to upgrade as there were caching fixes meanwhile.
>
> There is also possibility to disable caching in keycloak-server.json (or
> in standalone.xml in latest version). It's mentioned in the docs how to
> do it.
>
> Finally it may also help if you have opportunity to try with 2 Keycloak
> cluster nodes configured against same DB node. This may help to better
> isolate the problem and see if it's related to caching or to MariaDB
> cluster.
>
> Marek
>
> On 10/10/16 22:31, Josh Cain wrote:
> > Hi all,
> >
> > We're running into a problem with a couple of MariaDB instances +
> > Galera. When I go to add a client on the first Keycloak node/DB (we'll
> > call it DB01), it add successfully. I can then go to the second
> > Keycloak Node/DB (call this one DB02) and do not see the client on the
> > 'clients' list. However, if I were to add the same client on DB02, I
> > get the expected 'client with ID already exists' message. What's
more,
> > if I bounce the Keycloak node that talks to DB02, the client list
> > populates with the new entry added at DB01.
> >
> > Was guessing it's some kind of caching issue - is there a setting where
> > I can alter this behavior?
> >
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
10:08 a.m.
And are also both Keycloak nodes in the same infinispan cluster?
Marek
Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
We've got synchronous replication enabled. I've looked in the
DB
tables for both galera nodes and the data is there. e.g. both DB nodes
have client 'myclient' but the UI for Keycloak node 2 doesn't list a
'myclient'. But Keycloak will error if you try to add 'myclient'
saying it already exists.
On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
Then it's probably related to the Galera cluster rather then to
caching...
Do you have DB configured with synchronous replication (eg.
inserting some record on DB1 is successfully finished after the
record is successfully replicated to DB2 too) ?
You can maybe compare with the configuration in my docker image
https://github.com/mposolda/keycloak-mariadb
<https://github.com/mposolda/keycloak-mariadb> . I can't recall to
see any issue like this, but not sure about other aspects of my
configuration (performance etc).
Marek
On 12/10/16 19:08, Jared Blashka wrote:
> We're already running 1.9.8.Final. Our previous configuration was
> using 2 clustered nodes configured against the same DB node and
> we didn't run into this issue.
>
> On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>> wrote:
>
> Which Keycloak version are you using? If it's older than
> 1.9.8.Final,
> then it's suggested to upgrade as there were caching fixes
> meanwhile.
>
> There is also possibility to disable caching in
> keycloak-server.json (or
> in standalone.xml in latest version). It's mentioned in the
> docs how to
> do it.
>
> Finally it may also help if you have opportunity to try with
> 2 Keycloak
> cluster nodes configured against same DB node. This may help
> to better
> isolate the problem and see if it's related to caching or to
> MariaDB
> cluster.
>
> Marek
>
> On 10/10/16 22:31, Josh Cain wrote:
> > Hi all,
> >
> > We're running into a problem with a couple of MariaDB
> instances +
> > Galera. When I go to add a client on the first Keycloak
> node/DB (we'll
> > call it DB01), it add successfully. I can then go to the
> second
> > Keycloak Node/DB (call this one DB02) and do not see the
> client on the
> > 'clients' list. However, if I were to add the same client
> on DB02, I
> > get the expected 'client with ID already exists' message.
> What's more,
> > if I bounce the Keycloak node that talks to DB02, the
> client list
> > populates with the new entry added at DB01.
> >
> > Was guessing it's some kind of caching issue - is there a
> setting where
> > I can alter this behavior?
> >
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
> <mailto:keycloak-user@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-user
> <https://lists.jboss.org/mailman/listinfo/keycloak-user>
>
>
1:34 p.m.
Just to point out the maybe not so obvious - all realm configuration
including clients are cached in an Infinispan invalidation cache. I've got
no idea how to setup the Infinispan invalidation caches cross data centers,
but that would be required for entries to be re-loaded in one DC when
updated in another DC.
On 13 October 2016 at 17:08, Marek Posolda <mposolda(a)redhat.com> wrote:
And are also both Keycloak nodes in the same infinispan cluster?
Marek
Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
> We've got synchronous replication enabled. I've looked in the DB
> tables for both galera nodes and the data is there. e.g. both DB nodes
> have client 'myclient' but the UI for Keycloak node 2 doesn't list a
> 'myclient'. But Keycloak will error if you try to add 'myclient'
> saying it already exists.
>
> On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda(a)redhat.com
> <mailto:mposolda@redhat.com>> wrote:
>
> Then it's probably related to the Galera cluster rather then to
> caching...
>
> Do you have DB configured with synchronous replication (eg.
> inserting some record on DB1 is successfully finished after the
> record is successfully replicated to DB2 too) ?
>
> You can maybe compare with the configuration in my docker image
> https://github.com/mposolda/keycloak-mariadb
> <https://github.com/mposolda/keycloak-mariadb> . I can't recall to
> see any issue like this, but not sure about other aspects of my
> configuration (performance etc).
>
> Marek
>
>
> On 12/10/16 19:08, Jared Blashka wrote:
>> We're already running 1.9.8.Final. Our previous configuration was
>> using 2 clustered nodes configured against the same DB node and
>> we didn't run into this issue.
>>
>> On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
>> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>> wrote:
>>
>> Which Keycloak version are you using? If it's older than
>> 1.9.8.Final,
>> then it's suggested to upgrade as there were caching fixes
>> meanwhile.
>>
>> There is also possibility to disable caching in
>> keycloak-server.json (or
>> in standalone.xml in latest version). It's mentioned in the
>> docs how to
>> do it.
>>
>> Finally it may also help if you have opportunity to try with
>> 2 Keycloak
>> cluster nodes configured against same DB node. This may help
>> to better
>> isolate the problem and see if it's related to caching or to
>> MariaDB
>> cluster.
>>
>> Marek
>>
>> On 10/10/16 22:31, Josh Cain wrote:
>> > Hi all,
>> >
>> > We're running into a problem with a couple of MariaDB
>> instances +
>> > Galera. When I go to add a client on the first Keycloak
>> node/DB (we'll
>> > call it DB01), it add successfully. I can then go to the
>> second
>> > Keycloak Node/DB (call this one DB02) and do not see the
>> client on the
>> > 'clients' list. However, if I were to add the same client
>> on DB02, I
>> > get the expected 'client with ID already exists' message.
>> What's more,
>> > if I bounce the Keycloak node that talks to DB02, the
>> client list
>> > populates with the new entry added at DB01.
>> >
>> > Was guessing it's some kind of caching issue - is there a
>> setting where
>> > I can alter this behavior?
>> >
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user(a)lists.jboss.org
>> <mailto:keycloak-user@lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>> <https://lists.jboss.org/mailman/listinfo/keycloak-user>
>>
>>
>
>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user
1:54 p.m.
Both of our keycloak nodes are living in the same physical
datacenter+networking space and are the only two nodes in an infinispan
cluster; they're just each using a different Galera DB (and these are
clustered synchronously together along with a 3rd Galera node). We were
trying to validate that DB replication wouldn't break like it did for a
similar configuration we were using earlier (using asynchronous DB
replication). So the DB replication isn't breaking and appears to be
functioning as expected, but it looks like there's data cached by each
Keycloak node that doesn't get refreshed from the DB nor corrected by
infinispan. So far the only thing we've noticed are changes not appearing
in the Admin UI e.g. Realm/Client changes performed on Keycloak01 don't
appear in the UI for Keycloak02 but *do* appear in Galera02. The issue
doesn't seem to extend to client sessions; we haven't heard any issues of
people being asked to log in multiple times.
I'd be happy to run any specific tests in our set up if you want additional
info.
Jared
On Mon, Oct 17, 2016 at 2:34 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
Just to point out the maybe not so obvious - all realm configuration
including clients are cached in an Infinispan invalidation cache. I've got
no idea how to setup the Infinispan invalidation caches cross data centers,
but that would be required for entries to be re-loaded in one DC when
updated in another DC.
On 13 October 2016 at 17:08, Marek Posolda <mposolda(a)redhat.com> wrote:
> And are also both Keycloak nodes in the same infinispan cluster?
>
> Marek
>
> Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
> > We've got synchronous replication enabled. I've looked in the DB
> > tables for both galera nodes and the data is there. e.g. both DB nodes
> > have client 'myclient' but the UI for Keycloak node 2 doesn't list
a
> > 'myclient'. But Keycloak will error if you try to add
'myclient'
> > saying it already exists.
> >
> > On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda(a)redhat.com
> > <mailto:mposolda@redhat.com>> wrote:
> >
> > Then it's probably related to the Galera cluster rather then to
> > caching...
> >
> > Do you have DB configured with synchronous replication (eg.
> > inserting some record on DB1 is successfully finished after the
> > record is successfully replicated to DB2 too) ?
> >
> > You can maybe compare with the configuration in my docker image
> > https://github.com/mposolda/keycloak-mariadb
> > <https://github.com/mposolda/keycloak-mariadb> . I can't recall
to
> > see any issue like this, but not sure about other aspects of my
> > configuration (performance etc).
> >
> > Marek
> >
> >
> > On 12/10/16 19:08, Jared Blashka wrote:
> >> We're already running 1.9.8.Final. Our previous configuration was
> >> using 2 clustered nodes configured against the same DB node and
> >> we didn't run into this issue.
> >>
> >> On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
> >> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>> wrote:
> >>
> >> Which Keycloak version are you using? If it's older than
> >> 1.9.8.Final,
> >> then it's suggested to upgrade as there were caching fixes
> >> meanwhile.
> >>
> >> There is also possibility to disable caching in
> >> keycloak-server.json (or
> >> in standalone.xml in latest version). It's mentioned in the
> >> docs how to
> >> do it.
> >>
> >> Finally it may also help if you have opportunity to try with
> >> 2 Keycloak
> >> cluster nodes configured against same DB node. This may help
> >> to better
> >> isolate the problem and see if it's related to caching or to
> >> MariaDB
> >> cluster.
> >>
> >> Marek
> >>
> >> On 10/10/16 22:31, Josh Cain wrote:
> >> > Hi all,
> >> >
> >> > We're running into a problem with a couple of MariaDB
> >> instances +
> >> > Galera. When I go to add a client on the first Keycloak
> >> node/DB (we'll
> >> > call it DB01), it add successfully. I can then go to the
> >> second
> >> > Keycloak Node/DB (call this one DB02) and do not see the
> >> client on the
> >> > 'clients' list. However, if I were to add the same
client
> >> on DB02, I
> >> > get the expected 'client with ID already exists'
message.
> >> What's more,
> >> > if I bounce the Keycloak node that talks to DB02, the
> >> client list
> >> > populates with the new entry added at DB01.
> >> >
> >> > Was guessing it's some kind of caching issue - is there a
> >> setting where
> >> > I can alter this behavior?
> >> >
> >>
> >> _______________________________________________
> >> keycloak-user mailing list
> >> keycloak-user(a)lists.jboss.org
> >> <mailto:keycloak-user@lists.jboss.org>
> >> https://lists.jboss.org/mailman/listinfo/keycloak-user
> >> <https://lists.jboss.org/mailman/listinfo/keycloak-user>
> >>
> >>
> >
> >
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
2:55 p.m.
On 17/10/16 20:54, Jared Blashka wrote:
Both of our keycloak nodes are living in the same physical
datacenter+networking space and are the only two nodes in an
infinispan cluster; they're just each using a different Galera DB (and
these are clustered synchronously together along with a 3rd Galera
node). We were trying to validate that DB replication wouldn't break
like it did for a similar configuration we were using earlier (using
asynchronous DB replication). So the DB replication isn't breaking and
appears to be functioning as expected, but it looks like there's data
cached by each Keycloak node that doesn't get refreshed from the DB
nor corrected by infinispan. So far the only thing we've noticed are
changes not appearing in the Admin UI e.g. Realm/Client changes
performed on Keycloak01 don't appear in the UI for Keycloak02 but *do*
appear in Galera02. The issue doesn't seem to extend to client
sessions; we haven't heard any issues of people being asked to log in
multiple times.
I'd be happy to run any specific tests in our set up if you want
additional info.
Could you try this simple test like:
- Create user in admin console on keycloak node1
- Verify that user is visible on keycloak node2
- Then update this user on node2 (For example change his firstName)
- Go back to node1 and see if firstName was changed
This is the similar test, which I've tried with 2 keycloak cluster nodes
configured against 2 MariaDB Galera cluster nodes and it worked fine for
me. The automated test is here if you want to take a look :
https://github.com/mposolda/keycloak-mariadb/blob/master/mariadb-cluster-...
.
If this scenario works fine for you, then it's maybe just listing
clients, which is somehow broken. Then it's high probability that I will
reproduce in my environment too. Otherwise if user's scenario is broken
for you as well, then it's probably something related to your
environment setup though...
Marek
Jared
On Mon, Oct 17, 2016 at 2:34 PM, Stian Thorgersen <sthorger(a)redhat.com
<mailto:sthorger@redhat.com>> wrote:
Just to point out the maybe not so obvious - all realm
configuration including clients are cached in an Infinispan
invalidation cache. I've got no idea how to setup the Infinispan
invalidation caches cross data centers, but that would be required
for entries to be re-loaded in one DC when updated in another DC.
On 13 October 2016 at 17:08, Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
And are also both Keycloak nodes in the same infinispan cluster?
Marek
Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
> We've got synchronous replication enabled. I've looked in the DB
> tables for both galera nodes and the data is there. e.g.
both DB nodes
> have client 'myclient' but the UI for Keycloak node 2
doesn't list a
> 'myclient'. But Keycloak will error if you try to add
'myclient'
> saying it already exists.
>
> On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda
<mposolda(a)redhat.com <mailto:mposolda@redhat.com>
> <mailto:mposolda@redhat.com <mailto:mposolda@redhat.com>>>
wrote:
>
> Then it's probably related to the Galera cluster rather
then to
> caching...
>
> Do you have DB configured with synchronous replication (eg.
> inserting some record on DB1 is successfully finished
after the
> record is successfully replicated to DB2 too) ?
>
> You can maybe compare with the configuration in my
docker image
> https://github.com/mposolda/keycloak-mariadb
<https://github.com/mposolda/keycloak-mariadb>
> <https://github.com/mposolda/keycloak-mariadb
<https://github.com/mposolda/keycloak-mariadb>> . I can't
recall to
> see any issue like this, but not sure about other
aspects of my
> configuration (performance etc).
>
> Marek
>
>
> On 12/10/16 19:08, Jared Blashka wrote:
>> We're already running 1.9.8.Final. Our previous
configuration was
>> using 2 clustered nodes configured against the same DB
node and
>> we didn't run into this issue.
>>
>> On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
>> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>
<mailto:mposolda@redhat.com <mailto:mposolda@redhat.com>>> wrote:
>>
>> Which Keycloak version are you using? If it's older
than
>> 1.9.8.Final,
>> then it's suggested to upgrade as there were
caching fixes
>> meanwhile.
>>
>> There is also possibility to disable caching in
>> keycloak-server.json (or
>> in standalone.xml in latest version). It's
mentioned in the
>> docs how to
>> do it.
>>
>> Finally it may also help if you have opportunity to
try with
>> 2 Keycloak
>> cluster nodes configured against same DB node. This
may help
>> to better
>> isolate the problem and see if it's related to
caching or to
>> MariaDB
>> cluster.
>>
>> Marek
>>
>> On 10/10/16 22:31, Josh Cain wrote:
>> > Hi all,
>> >
>> > We're running into a problem with a couple of MariaDB
>> instances +
>> > Galera. When I go to add a client on the first
Keycloak
>> node/DB (we'll
>> > call it DB01), it add successfully. I can then
go to the
>> second
>> > Keycloak Node/DB (call this one DB02) and do not
see the
>> client on the
>> > 'clients' list. However, if I were to add the
same client
>> on DB02, I
>> > get the expected 'client with ID already exists'
message.
>> What's more,
>> > if I bounce the Keycloak node that talks to DB02, the
>> client list
>> > populates with the new entry added at DB01.
>> >
>> > Was guessing it's some kind of caching issue - is
there a
>> setting where
>> > I can alter this behavior?
>> >
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user(a)lists.jboss.org
<mailto:keycloak-user@lists.jboss.org>
>> <mailto:keycloak-user@lists.jboss.org
<mailto:keycloak-user@lists.jboss.org>>
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
<https://lists.jboss.org/mailman/listinfo/keycloak-user>
>> <https://lists.jboss.org/mailman/listinfo/keycloak-user
<https://lists.jboss.org/mailman/listinfo/keycloak-user>>
>>
>>
>
>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
<mailto:keycloak-user@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-user
<https://lists.jboss.org/mailman/listinfo/keycloak-user>
3:09 p.m.
I made a ReplUser on Keycloak01, was able to search for it and update it on
Keycloak02. Set the first name to "blah" using Keycloak02. Refreshed the
ReplUser page on Keycloak01 and I don't see a name change. If I search for
ReplUser on the search page on Keycloak01 I can see "blah" in the first
name column, but the first name inbox box is blank on the user details page.
Jared
On Mon, Oct 17, 2016 at 3:55 PM, Marek Posolda <mposolda(a)redhat.com> wrote:
On 17/10/16 20:54, Jared Blashka wrote:
Both of our keycloak nodes are living in the same physical
datacenter+networking space and are the only two nodes in an infinispan
cluster; they're just each using a different Galera DB (and these are
clustered synchronously together along with a 3rd Galera node). We were
trying to validate that DB replication wouldn't break like it did for a
similar configuration we were using earlier (using asynchronous DB
replication). So the DB replication isn't breaking and appears to be
functioning as expected, but it looks like there's data cached by each
Keycloak node that doesn't get refreshed from the DB nor corrected by
infinispan. So far the only thing we've noticed are changes not appearing
in the Admin UI e.g. Realm/Client changes performed on Keycloak01 don't
appear in the UI for Keycloak02 but *do* appear in Galera02. The issue
doesn't seem to extend to client sessions; we haven't heard any issues of
people being asked to log in multiple times.
I'd be happy to run any specific tests in our set up if you want
additional info.
Could you try this simple test like:
- Create user in admin console on keycloak node1
- Verify that user is visible on keycloak node2
- Then update this user on node2 (For example change his firstName)
- Go back to node1 and see if firstName was changed
This is the similar test, which I've tried with 2 keycloak cluster nodes
configured against 2 MariaDB Galera cluster nodes and it worked fine for
me. The automated test is here if you want to take a look :
https://github.com/mposolda/keycloak-mariadb/blob/master/
mariadb-cluster-test/src/test/java/org/keycloak/test/UsersClusterTest.java
.
If this scenario works fine for you, then it's maybe just listing clients,
which is somehow broken. Then it's high probability that I will reproduce
in my environment too. Otherwise if user's scenario is broken for you as
well, then it's probably something related to your environment setup
though...
Marek
Jared
On Mon, Oct 17, 2016 at 2:34 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
> Just to point out the maybe not so obvious - all realm configuration
> including clients are cached in an Infinispan invalidation cache. I've got
> no idea how to setup the Infinispan invalidation caches cross data centers,
> but that would be required for entries to be re-loaded in one DC when
> updated in another DC.
>
> On 13 October 2016 at 17:08, Marek Posolda <mposolda(a)redhat.com> wrote:
>
>> And are also both Keycloak nodes in the same infinispan cluster?
>>
>> Marek
>>
>> Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
>> > We've got synchronous replication enabled. I've looked in the DB
>> > tables for both galera nodes and the data is there. e.g. both DB nodes
>> > have client 'myclient' but the UI for Keycloak node 2 doesn't
list a
>> > 'myclient'. But Keycloak will error if you try to add
'myclient'
>> > saying it already exists.
>> >
>> > On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda(a)redhat.com
>> > <mailto:mposolda@redhat.com>> wrote:
>> >
>> > Then it's probably related to the Galera cluster rather then to
>> > caching...
>> >
>> > Do you have DB configured with synchronous replication (eg.
>> > inserting some record on DB1 is successfully finished after the
>> > record is successfully replicated to DB2 too) ?
>> >
>> > You can maybe compare with the configuration in my docker image
>> > https://github.com/mposolda/keycloak-mariadb
>> > <https://github.com/mposolda/keycloak-mariadb> . I can't
recall to
>> > see any issue like this, but not sure about other aspects of my
>> > configuration (performance etc).
>> >
>> > Marek
>> >
>> >
>> > On 12/10/16 19:08, Jared Blashka wrote:
>> >> We're already running 1.9.8.Final. Our previous configuration
was
>> >> using 2 clustered nodes configured against the same DB node and
>> >> we didn't run into this issue.
>> >>
>> >> On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
>> >> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>>
wrote:
>> >>
>> >> Which Keycloak version are you using? If it's older than
>> >> 1.9.8.Final,
>> >> then it's suggested to upgrade as there were caching fixes
>> >> meanwhile.
>> >>
>> >> There is also possibility to disable caching in
>> >> keycloak-server.json (or
>> >> in standalone.xml in latest version). It's mentioned in the
>> >> docs how to
>> >> do it.
>> >>
>> >> Finally it may also help if you have opportunity to try with
>> >> 2 Keycloak
>> >> cluster nodes configured against same DB node. This may help
>> >> to better
>> >> isolate the problem and see if it's related to caching or
to
>> >> MariaDB
>> >> cluster.
>> >>
>> >> Marek
>> >>
>> >> On 10/10/16 22:31, Josh Cain wrote:
>> >> > Hi all,
>> >> >
>> >> > We're running into a problem with a couple of MariaDB
>> >> instances +
>> >> > Galera. When I go to add a client on the first Keycloak
>> >> node/DB (we'll
>> >> > call it DB01), it add successfully. I can then go to the
>> >> second
>> >> > Keycloak Node/DB (call this one DB02) and do not see the
>> >> client on the
>> >> > 'clients' list. However, if I were to add the same
client
>> >> on DB02, I
>> >> > get the expected 'client with ID already exists'
message.
>> >> What's more,
>> >> > if I bounce the Keycloak node that talks to DB02, the
>> >> client list
>> >> > populates with the new entry added at DB01.
>> >> >
>> >> > Was guessing it's some kind of caching issue - is there
a
>> >> setting where
>> >> > I can alter this behavior?
>> >> >
>> >>
>> >> _______________________________________________
>> >> keycloak-user mailing list
>> >> keycloak-user(a)lists.jboss.org
>> >> <mailto:keycloak-user@lists.jboss.org>
>> >> https://lists.jboss.org/mailman/listinfo/keycloak-user
>> >> <https://lists.jboss.org/mailman/listinfo/keycloak-user>
>> >>
>> >>
>> >
>> >
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>
12:23 a.m.
Search for users aren't cached, but the user itself is. It sounds like you
don't have working Infinispan clustering setup and that the two nodes don't
see each other.
On 17 October 2016 at 22:09, Jared Blashka <jblashka(a)redhat.com> wrote:
I made a ReplUser on Keycloak01, was able to search for it and update
it
on Keycloak02. Set the first name to "blah" using Keycloak02. Refreshed the
ReplUser page on Keycloak01 and I don't see a name change. If I search for
ReplUser on the search page on Keycloak01 I can see "blah" in the first
name column, but the first name inbox box is blank on the user details page.
Jared
On Mon, Oct 17, 2016 at 3:55 PM, Marek Posolda <mposolda(a)redhat.com>
wrote:
> On 17/10/16 20:54, Jared Blashka wrote:
>
> Both of our keycloak nodes are living in the same physical
> datacenter+networking space and are the only two nodes in an infinispan
> cluster; they're just each using a different Galera DB (and these are
> clustered synchronously together along with a 3rd Galera node). We were
> trying to validate that DB replication wouldn't break like it did for a
> similar configuration we were using earlier (using asynchronous DB
> replication). So the DB replication isn't breaking and appears to be
> functioning as expected, but it looks like there's data cached by each
> Keycloak node that doesn't get refreshed from the DB nor corrected by
> infinispan. So far the only thing we've noticed are changes not appearing
> in the Admin UI e.g. Realm/Client changes performed on Keycloak01 don't
> appear in the UI for Keycloak02 but *do* appear in Galera02. The issue
> doesn't seem to extend to client sessions; we haven't heard any issues of
> people being asked to log in multiple times.
>
> I'd be happy to run any specific tests in our set up if you want
> additional info.
>
> Could you try this simple test like:
>
> - Create user in admin console on keycloak node1
> - Verify that user is visible on keycloak node2
> - Then update this user on node2 (For example change his firstName)
> - Go back to node1 and see if firstName was changed
>
> This is the similar test, which I've tried with 2 keycloak cluster nodes
> configured against 2 MariaDB Galera cluster nodes and it worked fine for
> me. The automated test is here if you want to take a look :
> https://github.com/mposolda/keycloak-mariadb/blob/master/mar
> iadb-cluster-test/src/test/java/org/keycloak/test/UsersClusterTest.java
> .
>
> If this scenario works fine for you, then it's maybe just listing
> clients, which is somehow broken. Then it's high probability that I will
> reproduce in my environment too. Otherwise if user's scenario is broken for
> you as well, then it's probably something related to your environment setup
> though...
>
> Marek
>
> Jared
>
> On Mon, Oct 17, 2016 at 2:34 PM, Stian Thorgersen <sthorger(a)redhat.com>
> wrote:
>
>> Just to point out the maybe not so obvious - all realm configuration
>> including clients are cached in an Infinispan invalidation cache. I've got
>> no idea how to setup the Infinispan invalidation caches cross data centers,
>> but that would be required for entries to be re-loaded in one DC when
>> updated in another DC.
>>
>> On 13 October 2016 at 17:08, Marek Posolda <mposolda(a)redhat.com> wrote:
>>
>>> And are also both Keycloak nodes in the same infinispan cluster?
>>>
>>> Marek
>>>
>>> Dne 12.10.2016 v 23:27 Jared Blashka napsal(a):
>>> > We've got synchronous replication enabled. I've looked in the
DB
>>> > tables for both galera nodes and the data is there. e.g. both DB nodes
>>> > have client 'myclient' but the UI for Keycloak node 2
doesn't list a
>>> > 'myclient'. But Keycloak will error if you try to add
'myclient'
>>> > saying it already exists.
>>> >
>>> > On Wed, Oct 12, 2016 at 4:42 PM, Marek Posolda <mposolda(a)redhat.com
>>> > <mailto:mposolda@redhat.com>> wrote:
>>> >
>>> > Then it's probably related to the Galera cluster rather then to
>>> > caching...
>>> >
>>> > Do you have DB configured with synchronous replication (eg.
>>> > inserting some record on DB1 is successfully finished after the
>>> > record is successfully replicated to DB2 too) ?
>>> >
>>> > You can maybe compare with the configuration in my docker image
>>> > https://github.com/mposolda/keycloak-mariadb
>>> > <https://github.com/mposolda/keycloak-mariadb> . I can't
recall
>>> to
>>> > see any issue like this, but not sure about other aspects of my
>>> > configuration (performance etc).
>>> >
>>> > Marek
>>> >
>>> >
>>> > On 12/10/16 19:08, Jared Blashka wrote:
>>> >> We're already running 1.9.8.Final. Our previous
configuration was
>>> >> using 2 clustered nodes configured against the same DB node and
>>> >> we didn't run into this issue.
>>> >>
>>> >> On Wed, Oct 12, 2016 at 2:45 AM, Marek Posolda
>>> >> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>>
wrote:
>>> >>
>>> >> Which Keycloak version are you using? If it's older
than
>>> >> 1.9.8.Final,
>>> >> then it's suggested to upgrade as there were caching
fixes
>>> >> meanwhile.
>>> >>
>>> >> There is also possibility to disable caching in
>>> >> keycloak-server.json (or
>>> >> in standalone.xml in latest version). It's mentioned in
the
>>> >> docs how to
>>> >> do it.
>>> >>
>>> >> Finally it may also help if you have opportunity to try
with
>>> >> 2 Keycloak
>>> >> cluster nodes configured against same DB node. This may
help
>>> >> to better
>>> >> isolate the problem and see if it's related to caching
or to
>>> >> MariaDB
>>> >> cluster.
>>> >>
>>> >> Marek
>>> >>
>>> >> On 10/10/16 22:31, Josh Cain wrote:
>>> >> > Hi all,
>>> >> >
>>> >> > We're running into a problem with a couple of
MariaDB
>>> >> instances +
>>> >> > Galera. When I go to add a client on the first
Keycloak
>>> >> node/DB (we'll
>>> >> > call it DB01), it add successfully. I can then go to
the
>>> >> second
>>> >> > Keycloak Node/DB (call this one DB02) and do not see
the
>>> >> client on the
>>> >> > 'clients' list. However, if I were to add the
same client
>>> >> on DB02, I
>>> >> > get the expected 'client with ID already
exists' message.
>>> >> What's more,
>>> >> > if I bounce the Keycloak node that talks to DB02, the
>>> >> client list
>>> >> > populates with the new entry added at DB01.
>>> >> >
>>> >> > Was guessing it's some kind of caching issue - is
there a
>>> >> setting where
>>> >> > I can alter this behavior?
>>> >> >
>>> >>
>>> >> _______________________________________________
>>> >> keycloak-user mailing list
>>> >> keycloak-user(a)lists.jboss.org
>>> >> <mailto:keycloak-user@lists.jboss.org>
>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>> >>
<https://lists.jboss.org/mailman/listinfo/keycloak-user>
>>> >>
>>> >>
>>> >
>>> >
>>>
>>> _______________________________________________
>>> keycloak-user mailing list
>>> keycloak-user(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>>
>>
>>
>
>
2782
days inactive
2790
days old
10 comments
4 participants
participants (4)
-
Jared Blashka -
Josh Cain -
Marek Posolda -
Stian Thorgersen