[keycloak-dev] Hangout this week to talk about cache and model splitting

[Bill Burke]

On 6/30/2014 6:39 AM, Stian Thorgersen wrote:
> I'd like to have a hangout this week to talk about cache, clustering and splitting of the model. I'd like to do this a.s.a.p. so we can decide on what we're going to do for 1.0.final.
>
> Last week I experimented with splitting the model into 3 parts: config, users and sessions. I've got something close to working at the moment. The basic idea is to create a new hybrid model provider that delegates to 3 different providers:
>

realm, roles, scopes, apps, clients, etc. are *NOT* config.  Please stop 
calling it config. :)

> * Config SPI - realms, apps, clients, roles and scope mappings
>    * Could quite easily do a json file implementation. However, what about clustering? Could we just reload the whole realm whenever it's changed?

This ties into caching as well.

>    * Not considering clustering, is there even any reason to store config in a database? Dropping support for DBs and Mongo for config would make things significantly simpler.

I don't agree having a flat file would make things simpler.
* You'd have to write a flat-file database that could handle session 
rollbacks.
* You'd have to require NFS or some other distributed file system for 
clustering.
* You've just added another storage mechanism the admin has to worry about.

>    * Can we load everything into mem on startup? How would that affect clustering?

That was already the plan with the caching layer I wrote.  As it was, 
the first access to a realm caches all non-user realm metadata.

>    * Should we add a revision to realms to make it easy to track consistency across servers in a cluster?
>    * In the long run we could move code from managers into KeycloakSession and HybridModelProvider (and also extract more shared code from the stores themselves)

95% of the model code is boilerplate.  THere is very little code that is 
duplicate.

>    * Does it make sense to have a read and read/write mode for config. For example admin uses read/write config, while logins and such use the read only config
>    * We could add a batch mode to the admin console. An admin can perform a number of changes that are kept in a draft version of the config on the server. Once the admin has done all the changes, he can then choose to review the changes through a page in the admin console and click push if he's happy with it. This could be taken further and have some users that are allowed to perform changes, but not push the changes.
>

IMO, those are very low priority compared to the other things we have to 
accomplish over the next year.

> * Users SPI
>    * Stores users, credentials and role mappings
>    * I expect this is the only one users of Keycloak could want to implement themselves
>    * What implementations do we provide? DBs, Mongo, LDAP?, Files?. Does it make sense to use PicketLink here (could provide jdbs, files, ldap, etc..)
>

With this separation, do we still need the AuthProvider SPI?

> * Sessions SPI
>    * Stores sessions (and probably login failures as well)
>    * In-mem implementation makes sense here. However, what about clustering?
>    * Do we still want JPA and Mongo implementations?
>

Without a JPA/Mongo implementation for User Sessions, you have to 
introduce another moving part in a cluster environment:  Infinispan, etc.

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com