[keycloak-dev] Hangout this week to talk about cache and model splitting

[Marek Posolda]

On 30.6.2014 18:45, Bill Burke wrote:
>
> On 6/30/2014 6:39 AM, Stian Thorgersen wrote:
>> I'd like to have a hangout this week to talk about cache, clustering and splitting of the model. I'd like to do this a.s.a.p. so we can decide on what we're going to do for 1.0.final.
>>
>> Last week I experimented with splitting the model into 3 parts: config, users and sessions. I've got something close to working at the moment. The basic idea is to create a new hybrid model provider that delegates to 3 different providers:
>>
> realm, roles, scopes, apps, clients, etc. are *NOT* config.  Please stop
> calling it config. :)
>
>> * Config SPI - realms, apps, clients, roles and scope mappings
>>     * Could quite easily do a json file implementation. However, what about clustering? Could we just reload the whole realm whenever it's changed?
> This ties into caching as well.
>
>>     * Not considering clustering, is there even any reason to store config in a database? Dropping support for DBs and Mongo for config would make things significantly simpler.
> I don't agree having a flat file would make things simpler.
> * You'd have to write a flat-file database that could handle session
> rollbacks.
> * You'd have to require NFS or some other distributed file system for
> clustering.
> * You've just added another storage mechanism the admin has to worry about.
I wonder if we can reconsider again to use Infinispan:-)

It has support for both clustering and transactions. Should address the 
issues you mentioned here. They have the SPIs to sync with persistence 
layers 
http://infinispan.org/docs/6.0.x/user_guide/user_guide.html#_persistence 
and eventually we can provide our own SPI implementations.

The secure communication among cluster nodes is possible to achieve at 
JGroups level by add support for ENCRYPT (encrypt cluster communication) 
and AUTH (authenticate cluster nodes before join cluster). Some details 
here http://www.jgroups.org/manual/html/protlist.html#Security . I 
looked at ENCRYPT and unfortunately it seems that it's not possible to 
secure the communication just with Keycloak itself, as it needs 
symmetric SecretKey for encode/decode communication between cluster 
nodes, when Keycloak provides private/public. SecretKey can be either 
read from JCEKS keystore file or automatically generated by first 
cluster node and then shared with other nodes with usage of asymmetric 
private/public. It seems that with some tweaks we can use Keycloak keys 
here.

I can investigate it more, if you see Infinispan as an option.
>
>>     * Can we load everything into mem on startup? How would that affect clustering?
> That was already the plan with the caching layer I wrote.  As it was,
> the first access to a realm caches all non-user realm metadata.
>
>>     * Should we add a revision to realms to make it easy to track consistency across servers in a cluster?
>>     * In the long run we could move code from managers into KeycloakSession and HybridModelProvider (and also extract more shared code from the stores themselves)
> 95% of the model code is boilerplate.  THere is very little code that is
> duplicate.
>
>>     * Does it make sense to have a read and read/write mode for config. For example admin uses read/write config, while logins and such use the read only config
>>     * We could add a batch mode to the admin console. An admin can perform a number of changes that are kept in a draft version of the config on the server. Once the admin has done all the changes, he can then choose to review the changes through a page in the admin console and click push if he's happy with it. This could be taken further and have some users that are allowed to perform changes, but not push the changes.
>>
> IMO, those are very low priority compared to the other things we have to
> accomplish over the next year.
>
>> * Users SPI
>>     * Stores users, credentials and role mappings
>>     * I expect this is the only one users of Keycloak could want to implement themselves
>>     * What implementations do we provide? DBs, Mongo, LDAP?, Files?. Does it make sense to use PicketLink here (could provide jdbs, files, ldap, etc..)
>>
> With this separation, do we still need the AuthProvider SPI?
I would say yes as implementing AuthProvider is still much easier then 
implementing whole users storage. And we already have some community 
members who implemented their own AuthenticationProviders.

And for example LDAP is good just for authentication, but much more 
tricky for save all other things related to UserModel (for example 
attributes, requiredActions etc) due to static nature of LDAP schema.

Marek
>
>> * Sessions SPI
>>     * Stores sessions (and probably login failures as well)
>>     * In-mem implementation makes sense here. However, what about clustering?
>>     * Do we still want JPA and Mongo implementations?
>>
> Without a JPA/Mongo implementation for User Sessions, you have to
> introduce another moving part in a cluster environment:  Infinispan, etc.
>