[keycloak-user] Shared datastore?
bauer_marie at gmx.net
bauer_marie at gmx.net
Mon Jan 21 01:15:43 EST 2019
Hi,
we are also using a shared database store for the sessions and until Keycloak 3.0.0 (Infinispant 8.1.0) it worked with passivation disabled.
Lately we updated to 3.4.3 (Infinispan 8.2.) with the same configuration, but it's not working anymore. Sessions are lost and not written to the database (work table works and gets entries on startup).
the configuration looks like that:
<distributed-cache name="sessions" mode="SYNC" owners="2">
<string-keyed-jdbc-store shared="true" preload="true" fetch-state="false" purge="false" passivation="false" data-source="KeycloakSessionDS">
<string-keyed-table prefix="ispn_entry">
<id-column name="id_column" type="VARCHAR(255)" />
<data-column name="data_column" type="BLOB" />
<timestamp-column name="timestamp_column" type="BIGINT" />
</string-keyed-table>
</string-keyed-jdbc-store>
</distributed-cache>
<distributed-cache name="offlineSessions" mode="SYNC" owners="2">
<string-keyed-jdbc-store shared="true" preload="true" fetch-state="false" purge="false" data-source="KeycloakSessionDS">
<string-keyed-table prefix="ispn_entry">
<id-column name="id_column" type="VARCHAR(255)" />
<data-column name="data_column" type="BLOB" />
<timestamp-column name="timestamp_column" type="BIGINT" />
</string-keyed-table>
</string-keyed-jdbc-store>
</distributed-cache>
<distributed-cache name="loginFailures" mode="SYNC" owners="2"/>
<replicated-cache name="work" mode="SYNC">
<string-keyed-jdbc-store shared="true" preload="true" fetch-state="false" purge="false" data-source="KeycloakSessionDS">
<string-keyed-table prefix="ispn_entry">
<id-column name="id_column" type="VARCHAR(255)" />
<data-column name="data_column" type="BLOB" />
<timestamp-column name="timestamp_column" type="BIGINT" />
</string-keyed-table>
</string-keyed-jdbc-store>
</replicated-cache>
Is there any configuration that I could change to make it work again? Will it work in future versions?
Marie
Gesendet: Donnerstag, 15. November 2018 um 19:38 Uhr
Von: "Nicolas Ocquidant" <nocquidant at gmail.com>
An: mposolda at redhat.com
Cc: keycloak-user at lists.jboss.org, wburns at redhat.com
Betreff: Re: [keycloak-user] Shared datastore?
OK, so I setup one KC node with a remote store, and one JDG (ISPN) server
with a shared JDBC store and it seems to work fine. Thanks!
I would have preferred not to add a JDG server but at least I have a
solution.
But a few things remain unclear:
- each time I update/insert/delete a session entity from KC, the JDG
server will always be updated, right?
- what about read operations? Does it mean there are 2 independent
caches, one in KC and one in ISPN (JMX shows same numEntriesInMemory for
both)?
So let's say I am not interested in Cross DC yet, but i want to set up a
cluster of 3 VMs (with HAProxy in front of them), each VM holding one KC
process + one ISPN process:
* VM#1 : kc_node11 && ispn_node11 (kc_node11 uses ispn_node11 as remote
cache)
* VM#2 : kc_node21 && ispn_node21 (kc_node21 uses ispn_node21 as remote
cache)
* VM#3 : kc_node31 && ispn_node31 (kc_node31 uses ispn_node31 as remote
cache)
Which cache mode
<http://infinispan.org/docs/stable/user_guide/user_guide.html#which_cache_mode_should_i_use>
should I use then and how many owners, for both KC and ISPN? Invalidation
mode and numOfOwner=2 for KC, and distributed mode and numOfOwner=2 for
ISPN? Would that work?
Thanks again
--nick
Le jeu. 15 nov. 2018 à 11:55, Marek Posolda <mposolda at redhat.com> a écrit :
> Yes, true. We're using SKIP_CACHE_STORE when writing to sessions. We never
> tested with CacheStores enabled.
>
> The only store, which we're tested with, is the "remote-store" which we're
> using for the cross-datacenter setup. We have lots of places when we're not
> just writing data to the "cache" directly and let the "remote-store" to
> propagate it, but instead we obtain "remoteCache" instance from the
> underlying remote-store and CRUD data directly to remoteCache to have some
> optimizations and guaranteed consistency and atomicity for remoteCache
> operations (EG. putIfAbsent, replace etc). That's also the reason why we're
> using SKIP_CACHE_STORE flag.
>
> Feel free to create JIRA for better support of other CacheStores.
>
> The other possibility to workaround this (besides what Sebastian already
> mentioned) is to have JDG server and configure your cache with the
> remote-store as described in our "Cross-Datacenter setup" documentation. On
> JDG side, you can configure the JDBC store to your cache. In other words,
> the session will be always written to JDG and JDG will write it to the
> undrlying JDBC. I know this option is far from ideal (you need to add JDG
> server just to workaround things), just mentioning it for completeness.
>
> Marek
>
>
> On 09/11/18 14:29, Sebastian Laskawiec wrote:
>
> Yes, I think that could be case, I see a plenty of places where we
> use SKIP_CACHE_STORE.
>
> Let me ask Marek for help here since it has been implemented long before I
> joined the team and I don't know the history behind it...
>
> On Thu, Nov 8, 2018 at 8:48 PM William Burns <wburns at redhat.com> wrote:
>
>>
>>
>> ----- Original Message -----
>> > From: "Sebastian Laskawiec" <slaskawi at redhat.com>
>> > To: "Nicolas Ocquidant" <nocquidant at gmail.com>
>> > Cc: keycloak-user at lists.jboss.org, "Will Burns Rosenquist Burns" <
>> wburns at redhat.com>
>> > Sent: Thursday, November 8, 2018 12:33:47 PM
>> > Subject: Re: [keycloak-user] Shared datastore?
>> >
>> > So I think there are at least two ways to address this problem. This
>> first
>> > one is to use Offline Tokens [1]. I'm not sure if that fits into your
>> > application since it requires your client applications to store the
>> token.
>> > In other words you can simply delegate this problem one layer below in
>> your
>> > system.
>> >
>> > If that doesn't work for you, yes passivation is a way to go. Frankly, I
>> > haven't used passivation but from the manual I see it works hand in hand
>> > with eviction [2][3]. Will (on CC) can probably correct me here, but my
>> > understanding is that whenever an entry gets evicted, the passivation
>> > mechanism picks it up and stores somewhere.
>>
>> It does and it works, the problem is that passivation doesn't play well
>> with shared stores in Infinispan. We prevent this configuration in 9.4 or
>> newer even.
>>
>> I recommended that Nicolas just use eviction and a shared store without
>> passivation. However it seems that entries are not written to the store in
>> this configuration. My guess is that KeyCloak performs write operations
>> with the SKIP_CACHE_STORE flag and assumes entries will only be written to
>> the store due to passivation. Is there a reason for that?
>>
>> >
>> > [1] http://blog.keycloak.org/2015/12/offline-tokens-in-keycloak.html[http://blog.keycloak.org/2015/12/offline-tokens-in-keycloak.html]
>> > [2]
>> >
>> http://infinispan.org/docs/stable/user_guide/user_guide.html#cache_passivation[http://infinispan.org/docs/stable/user_guide/user_guide.html#cache_passivation]
>> > [3]
>> >
>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/eviction/impl/EvictionWithPassivationTest.java#L61-L69[https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/eviction/impl/EvictionWithPassivationTest.java#L61-L69]
>> >
>> > On Thu, Nov 8, 2018 at 5:40 PM Nicolas Ocquidant <nocquidant at gmail.com>
>> > wrote:
>> >
>> > > My requirements are the following: store tokens emitted by KC during
>> one
>> > > year.
>> > >
>> > > I don't know how many users there are, but here are the number I get:
>> > > * the number of connections a week is about 700k.
>> > > * the number of session refresh a week is about 200k.
>> > >
>> > > I approximated around 1M of sessions a week, thus 52M a year.
>> > > In memory, a user session has been estimated around 4KB (about 1KB in
>> > > file/DB).
>> > >
>> > > But I guess a refresh does not create another session isn't it? And
>> maybe
>> > > it's possible to ask KC to delete previous emitted tokens when a new
>> one is
>> > > created for a same user?
>> > >
>> > > If yes, my estimation is probably a little bit too high here, but I
>> > > certainly have several millions of tokens to keep (and maybe dozens of
>> > > millions).
>> > >
>> > > Thanks
>> > > --nick
>> > >
>> > > Le mer. 7 nov. 2018 à 18:17, Nicolas Ocquidant <nocquidant at gmail.com>
>> a
>> > > écrit :
>> > >
>> > > > Hi,
>> > > >
>> > > > According to Infinispan, when passivation is disabled, every update
>> to
>> > > the
>> > > > cache should always write to the store.
>> > > >
>> > > > But I can't manage to get it work with Keycloak. If I disable
>> > > passivation,
>> > > > my SQL store (Postgres) stays empty, even if the cache is full.
>> > > >
>> > > > So, if passivation is needed for Keycloak to write to the DB, it
>> means
>> > > > that the use of a shared DB is not possible...
>> > > >
>> > > > But this leads to another issue for me. Enable passivation without a
>> > > > shared DB seems to imply that either 'fetch-state' or 'purge'
>> should be
>> > > > enabled on startup, in order for the cache to not contain stale
>> entries.
>> > > >
>> > > > 15:27:44,626 WARN
>> > > >
>> [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder]
>> > > (MSC
>> > > > service thread 1-6) ISPN000149: Fetch persistent state and purge on
>> > > startup
>> > > > are both disabled, cache may contain stale entries on startup
>> > > >
>> > > > As I need to keep millions of sessions, this will considerably slow
>> down
>> > > > the startup of my node (when started again after a crash for
>> instance).
>> > > >
>> > > > So, is shared datastore allowed in Keycloak? If yes, how to enable
>> it?
>> > > > Otherwise what other options do I have to improve my startup time,
>> if
>> > > > millions of sessions are in the store?
>> > > >
>> > > > Thanks
>> > > > --nick
>> > > >
>> > > _______________________________________________
>> > > keycloak-user mailing list
>> > > keycloak-user at lists.jboss.org
>> > > https://lists.jboss.org/mailman/listinfo/keycloak-user[https://lists.jboss.org/mailman/listinfo/keycloak-user]
>> >
>>
>
>
_______________________________________________
keycloak-user mailing list
keycloak-user at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user[https://lists.jboss.org/mailman/listinfo/keycloak-user]
More information about the keycloak-user
mailing list