[keycloak-user] Shared datastore?

[William Burns]

----- Original Message -----
> From: "Sebastian Laskawiec" <slaskawi at redhat.com>
> To: "Nicolas Ocquidant" <nocquidant at gmail.com>
> Cc: keycloak-user at lists.jboss.org, "Will Burns Rosenquist Burns" <wburns at redhat.com>
> Sent: Thursday, November 8, 2018 12:33:47 PM
> Subject: Re: [keycloak-user] Shared datastore?
> 
> So I think there are at least two ways to address this problem. This first
> one is to use Offline Tokens [1]. I'm not sure if that fits into your
> application since it requires your client applications to store the token.
> In other words you can simply delegate this problem one layer below in your
> system.
> 
> If that doesn't work for you, yes passivation is a way to go. Frankly, I
> haven't used passivation but from the manual I see it works hand in hand
> with eviction [2][3]. Will (on CC) can probably correct me here, but my
> understanding is that whenever an entry gets evicted, the passivation
> mechanism picks it up and stores somewhere.

It does and it works, the problem is that passivation doesn't play well with shared stores in Infinispan. We prevent this configuration in 9.4 or newer even.

I recommended that Nicolas just use eviction and a shared store without passivation. However it seems that entries are not written to the store in this configuration. My guess is that KeyCloak performs write operations with the SKIP_CACHE_STORE flag and assumes entries will only be written to the store due to passivation. Is there a reason for that?

> 
> [1] http://blog.keycloak.org/2015/12/offline-tokens-in-keycloak.html
> [2]
> http://infinispan.org/docs/stable/user_guide/user_guide.html#cache_passivation
> [3]
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/eviction/impl/EvictionWithPassivationTest.java#L61-L69
> 
> On Thu, Nov 8, 2018 at 5:40 PM Nicolas Ocquidant <nocquidant at gmail.com>
> wrote:
> 
> > My requirements are the following: store tokens emitted by KC during one
> > year.
> >
> > I don't know how many users there are, but here are the number I get:
> >   * the number of connections a week is about 700k.
> >   * the number of session refresh a week is about 200k.
> >
> > I approximated around 1M of sessions a week, thus 52M a year.
> > In memory, a user session has been estimated around 4KB (about 1KB in
> > file/DB).
> >
> > But I guess a refresh does not create another session isn't it? And maybe
> > it's possible to ask KC to delete previous emitted tokens when a new one is
> > created for a same user?
> >
> > If yes, my estimation is probably a little bit too high here, but I
> > certainly have several millions of tokens to keep (and maybe dozens of
> > millions).
> >
> > Thanks
> > --nick
> >
> > Le mer. 7 nov. 2018 à 18:17, Nicolas Ocquidant <nocquidant at gmail.com> a
> > écrit :
> >
> > > Hi,
> > >
> > > According to Infinispan, when passivation is disabled, every update to
> > the
> > > cache should always write to the store.
> > >
> > > But I can't manage to get it work with Keycloak. If I disable
> > passivation,
> > > my SQL store (Postgres) stays empty, even if the cache is full.
> > >
> > > So, if passivation is needed for Keycloak to write to the DB, it means
> > > that the use of a shared DB is not possible...
> > >
> > > But this leads to another issue for me. Enable passivation without a
> > > shared DB seems to imply that either 'fetch-state' or 'purge' should be
> > > enabled on startup, in order for the cache to not contain stale entries.
> > >
> > > 15:27:44,626 WARN
> > > [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder]
> > (MSC
> > > service thread 1-6) ISPN000149: Fetch persistent state and purge on
> > startup
> > > are both disabled, cache may contain stale entries on startup
> > >
> > > As I need to keep millions of sessions, this will considerably slow down
> > > the startup of my node (when started again after a crash for instance).
> > >
> > > So, is shared datastore allowed in Keycloak? If yes, how to enable it?
> > > Otherwise what other options do I have to improve my startup time, if
> > > millions of sessions are in the store?
> > >
> > > Thanks
> > > --nick
> > >
> > _______________________________________________
> > keycloak-user mailing list
> > keycloak-user at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-user
>