[keycloak-user] Shared datastore?

[Sebastian Laskawiec]

Yes, I think that could be case, I see a plenty of places where we
use SKIP_CACHE_STORE.

Let me ask Marek for help here since it has been implemented long before I
joined the team and I don't know the history behind it...

On Thu, Nov 8, 2018 at 8:48 PM William Burns <wburns at redhat.com> wrote:

>
>
> ----- Original Message -----
> > From: "Sebastian Laskawiec" <slaskawi at redhat.com>
> > To: "Nicolas Ocquidant" <nocquidant at gmail.com>
> > Cc: keycloak-user at lists.jboss.org, "Will Burns Rosenquist Burns" <
> wburns at redhat.com>
> > Sent: Thursday, November 8, 2018 12:33:47 PM
> > Subject: Re: [keycloak-user] Shared datastore?
> >
> > So I think there are at least two ways to address this problem. This
> first
> > one is to use Offline Tokens [1]. I'm not sure if that fits into your
> > application since it requires your client applications to store the
> token.
> > In other words you can simply delegate this problem one layer below in
> your
> > system.
> >
> > If that doesn't work for you, yes passivation is a way to go. Frankly, I
> > haven't used passivation but from the manual I see it works hand in hand
> > with eviction [2][3]. Will (on CC) can probably correct me here, but my
> > understanding is that whenever an entry gets evicted, the passivation
> > mechanism picks it up and stores somewhere.
>
> It does and it works, the problem is that passivation doesn't play well
> with shared stores in Infinispan. We prevent this configuration in 9.4 or
> newer even.
>
> I recommended that Nicolas just use eviction and a shared store without
> passivation. However it seems that entries are not written to the store in
> this configuration. My guess is that KeyCloak performs write operations
> with the SKIP_CACHE_STORE flag and assumes entries will only be written to
> the store due to passivation. Is there a reason for that?
>
> >
> > [1] http://blog.keycloak.org/2015/12/offline-tokens-in-keycloak.html
> > [2]
> >
> http://infinispan.org/docs/stable/user_guide/user_guide.html#cache_passivation
> > [3]
> >
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/eviction/impl/EvictionWithPassivationTest.java#L61-L69
> >
> > On Thu, Nov 8, 2018 at 5:40 PM Nicolas Ocquidant <nocquidant at gmail.com>
> > wrote:
> >
> > > My requirements are the following: store tokens emitted by KC during
> one
> > > year.
> > >
> > > I don't know how many users there are, but here are the number I get:
> > >   * the number of connections a week is about 700k.
> > >   * the number of session refresh a week is about 200k.
> > >
> > > I approximated around 1M of sessions a week, thus 52M a year.
> > > In memory, a user session has been estimated around 4KB (about 1KB in
> > > file/DB).
> > >
> > > But I guess a refresh does not create another session isn't it? And
> maybe
> > > it's possible to ask KC to delete previous emitted tokens when a new
> one is
> > > created for a same user?
> > >
> > > If yes, my estimation is probably a little bit too high here, but I
> > > certainly have several millions of tokens to keep (and maybe dozens of
> > > millions).
> > >
> > > Thanks
> > > --nick
> > >
> > > Le mer. 7 nov. 2018 à 18:17, Nicolas Ocquidant <nocquidant at gmail.com>
> a
> > > écrit :
> > >
> > > > Hi,
> > > >
> > > > According to Infinispan, when passivation is disabled, every update
> to
> > > the
> > > > cache should always write to the store.
> > > >
> > > > But I can't manage to get it work with Keycloak. If I disable
> > > passivation,
> > > > my SQL store (Postgres) stays empty, even if the cache is full.
> > > >
> > > > So, if passivation is needed for Keycloak to write to the DB, it
> means
> > > > that the use of a shared DB is not possible...
> > > >
> > > > But this leads to another issue for me. Enable passivation without a
> > > > shared DB seems to imply that either 'fetch-state' or 'purge' should
> be
> > > > enabled on startup, in order for the cache to not contain stale
> entries.
> > > >
> > > > 15:27:44,626 WARN
> > > >
> [org.infinispan.configuration.cache.AbstractStoreConfigurationBuilder]
> > > (MSC
> > > > service thread 1-6) ISPN000149: Fetch persistent state and purge on
> > > startup
> > > > are both disabled, cache may contain stale entries on startup
> > > >
> > > > As I need to keep millions of sessions, this will considerably slow
> down
> > > > the startup of my node (when started again after a crash for
> instance).
> > > >
> > > > So, is shared datastore allowed in Keycloak? If yes, how to enable
> it?
> > > > Otherwise what other options do I have to improve my startup time, if
> > > > millions of sessions are in the store?
> > > >
> > > > Thanks
> > > > --nick
> > > >
> > > _______________________________________________
> > > keycloak-user mailing list
> > > keycloak-user at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/keycloak-user
> >
>