[keycloak-dev] Hangout this week to talk about cache and model splitting
Stian Thorgersen
stian at redhat.com
Tue Jul 1 04:47:21 EDT 2014
----- Original Message -----
> From: "Bill Burke" <bburke at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Monday, 30 June, 2014 5:45:20 PM
> Subject: Re: [keycloak-dev] Hangout this week to talk about cache and model splitting
>
>
>
> On 6/30/2014 6:39 AM, Stian Thorgersen wrote:
> > I'd like to have a hangout this week to talk about cache, clustering and
> > splitting of the model. I'd like to do this a.s.a.p. so we can decide on
> > what we're going to do for 1.0.final.
> >
> > Last week I experimented with splitting the model into 3 parts: config,
> > users and sessions. I've got something close to working at the moment. The
> > basic idea is to create a new hybrid model provider that delegates to 3
> > different providers:
> >
>
> realm, roles, scopes, apps, clients, etc. are *NOT* config. Please stop
> calling it config. :)
Realms SPI?
>
> > * Config SPI - realms, apps, clients, roles and scope mappings
> > * Could quite easily do a json file implementation. However, what about
> > clustering? Could we just reload the whole realm whenever it's changed?
>
> This ties into caching as well.
>
> > * Not considering clustering, is there even any reason to store config
> > in a database? Dropping support for DBs and Mongo for config would make
> > things significantly simpler.
>
> I don't agree having a flat file would make things simpler.
> * You'd have to write a flat-file database that could handle session
> rollbacks.
> * You'd have to require NFS or some other distributed file system for
> clustering.
> * You've just added another storage mechanism the admin has to worry about.
I was thinking something a lot simpler. A single keycloak-realms.json file, and whenever anything is changed the whole file is written again. Thinking about it some more it only makes sense in smaller deployments, where there's only one or two realms, a few apps and a few roles. So we'd still want DBs.
We could add this quite easily as we already need to have the capability of importing/exporting the whole db to a json file. It could just have a limitation that it doesn't do clustering, so would only be aimed for test/dev and small deployments.
>
> > * Can we load everything into mem on startup? How would that affect
> > clustering?
>
> That was already the plan with the caching layer I wrote. As it was,
> the first access to a realm caches all non-user realm metadata.
Basically what I was wondering here was if we could eagerly fetch everything for a realm and just load everything in mem in one go. Also, does it make sense to load it at startup rather than on first visit?
>
> > * Should we add a revision to realms to make it easy to track
> > consistency across servers in a cluster?
> > * In the long run we could move code from managers into KeycloakSession
> > and HybridModelProvider (and also extract more shared code from the
> > stores themselves)
>
> 95% of the model code is boilerplate. THere is very little code that is
> duplicate.
There's still 5% ;)
>
> > * Does it make sense to have a read and read/write mode for config. For
> > example admin uses read/write config, while logins and such use the
> > read only config
> > * We could add a batch mode to the admin console. An admin can perform a
> > number of changes that are kept in a draft version of the config on the
> > server. Once the admin has done all the changes, he can then choose to
> > review the changes through a page in the admin console and click push
> > if he's happy with it. This could be taken further and have some users
> > that are allowed to perform changes, but not push the changes.
> >
>
> IMO, those are very low priority compared to the other things we have to
> accomplish over the next year.
Totally agree, those where just some ideas that popped into my head.
>
> > * Users SPI
> > * Stores users, credentials and role mappings
> > * I expect this is the only one users of Keycloak could want to
> > implement themselves
> > * What implementations do we provide? DBs, Mongo, LDAP?, Files?. Does it
> > make sense to use PicketLink here (could provide jdbs, files, ldap,
> > etc..)
> >
>
> With this separation, do we still need the AuthProvider SPI?
AuthProvider is trying to achieve something different than storing users, and the combination of the two is quite flexible. Someone could still achieve what they can achieve with AuthProvider by creating a UserProvider impl that delegates to another UserProvider impl for storage, so maybe we could respin the AuthProvider SPI into a UserProvider that can use another UserProvider for storing users, and one or more UserProviders for authentication. That way we'd still have the capability, but with it less hard-coded into the core code.
+--------------------+
Store | |
+------------> JpaUserProvider |
| | |
| +--------------------+
+------------------+
| |
| AuthUserProvider | +--------------------+
| | | |
+------------------+ | LDAPUserProvider |
+------------> |
Authenticate +--------------------+
Same again with Sync SPI, that could be done as a delegating UserProvider store.
+--------------------+
Store | |
+------------> JpaUserProvider |
| | |
| +--------------------+
+------------------+
| |
| SyncUserProvider | +--------------------+
| | | |
+------------------+ | LDAPUserProvider |
+------------> |
Sync +--------------------+
>
> > * Sessions SPI
> > * Stores sessions (and probably login failures as well)
> > * In-mem implementation makes sense here. However, what about
> > clustering?
> > * Do we still want JPA and Mongo implementations?
> >
>
> Without a JPA/Mongo implementation for User Sessions, you have to
> introduce another moving part in a cluster environment: Infinispan, etc.
Yep, for 1.0.final we should do JPA/Mongo. Then in the future consider Infinispan or somethin'.
I've added a very simple in-mem session store though that can be used in non-clustered mode and test/dev.
>
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
More information about the keycloak-dev
mailing list