[keycloak-dev] in-memory only federated users

[Marek Posolda]

On 03/12/15 16:57, Bill Burke wrote:
> Either we redo the federation SPI or work with the current one.
>
> It is just not as simple as you state.  You can't just chain in a 
> generic InMemoryProvider.  Federation needs to be able to proxy the 
> UserModel so that it can handle write methods if it wants to. Or 
> delegate lookup of certain things to LDAP. 
I am not seeing why it's an issue? The InMemory will be kind of same 
thing like currently JPA. It just won't store the things into database, 
but into memory. That's the only difference. It will just be the 
provider at the end of the chain. UserFederationManager can proxy users 
exactly like now and doesn't require any code changes.

So when admin configure:

userFederationMAnager => inMemory

The call of "session.userStorage()" from UserFederationManager will 
return underlying InMemory instead of current JPA.

> Also, UserFederationManager has to be first in the chain so that if 
> something is found in cache, it can let the federation provider proxy 
> the cache if it wants to.
That's not a problem as well.

What I mean is the flexibility to configure things exactly how you want 
for various cases.

3 basic setups:

1. userFederation => cache => JPA

This is what we have now and it will be the default setup.  It 's useful 
for deployments when admins are often doing changes directly in their 
LDAP and they want the change imediatelly visible in Keycloak. So the 
UserFederationProvider proxy is always the top and when you call:

user.getFirstName()

you will retrieve the firstName from LDAP. The disadvantage of this 
setup is performance (each HTTP request needs to query LDAP like it's now)


2. cache => userFederation => JPA

This will be useful for deployments when amins are not doing changes 
directly in their LDAP. So once you retrieve the user from 
LDAP+KeycloakDB, you will save him to cache and call to:

user.getFirstName()

will always return the value from cache. Hence when admin changes 
directly in LDAP, it won't be immediately visible in Keycloak.

But on the other hand update from Keycloak to LDAP is not an issue. When 
you call:

user.setFirstName("foo")

the cache will call getDelegateForUpdate (exactly like now) and it will 
return proxy object, so the saving of firstName is propagated to LDAP 
(if it's writable) and to Keycloak JPA DB as well.


3. userFederation => inMemory

The federation backed by pure in-memory storage. The federation proxy is 
on top, writing and reading to/from LDAP is not a problem and has 
preference over the content from memory. The only difference from (1) is 
that underlying backend is pure memory (infinispan) instead of JPA DB

There is also alternative to use combination of 2 and 3:
cache => userFederation => inMemory

etc etc.


I can see this as most flexible approach without dependencies of various 
providers on each other.

Marek
>
> What we need is a special interface for the cache:
>
> cache.cacheUser(UserModel user);
>
> The cache would also work with UserFederationManager rather than a 
> generic UserProvider. UserFederationManager would gain methods like:
> UserFederationManager.getUncachedUserById() which the cache would 
> invoke.  UserFederationManager would break down the user id and either 
> know it was local storage or something that would have to be delegated 
> to a UserProvider.
>
>
>
>
>
> On 12/3/2015 10:32 AM, Marek Posolda wrote:
>> IMO the more important use-case that in-memory federated users is the
>> caching of federated users.
>>
>> Currently if you call: session.users().getUserById() and the user with
>> ID "123" is LDAP (or other federationProvider) user, there is always
>> call to UserFederationProvider.validateAndProxy , which results in LDAP
>> query.
>>
>> If we introduce the chaining of UserProvider (something I already
>> proposed earlier), you can switch UserFederationProvider with cache, so
>> you will have:
>> cache => userFederationManager => JPA
>>
>> instead of current:
>>
>> userFederationManager => cache => JPA
>>
>>
>> With that in mind, we can easily implement in-memory as another
>> implementation of UserProvider, which will hold users purely in-memory.
>> Our current DefaultCacheUserProvider always require delegate to call
>> write operations. But this in-memory provider would be something
>> different. It won't use any delegate as it will be in the end of the
>> chain. So for in-memory federation you will just configure:
>>
>> userFederationManager => inMemoryProvider
>>
>> and you're done. No needs for special ID handling or something like 
>> that.
>>
>> With chaining of UserProvider we have biggest flexibility for various
>> needs IMO. That's why I would rather go this way TBH.
>> Marek
>>
>> On 02/12/15 17:48, Bill Burke wrote:
>>> I'm looking into in-memory only no-import federated users.  What we
>>> would want to do is allow the UserFederationProvider to create an
>>> in-memory UserModel and allow for that UserModel to be cached via our
>>> current architecture.
>>>
>>> The current design assumes that all federated users are imported.  This
>>> includes our caching layer too!  To add to that, the user isn't cached
>>> until the 2nd request i.e.:
>>>
>>> 1. username/password page would hit the UserFederationProvider and the
>>> user would be imported into Keycloak.  This imported user is not 
>>> cached,
>>> only imported into the database for this request's KeycloakSession
>>> 2. OTP Page or code 2 token would then want to lookup the user by id as
>>> that is what is stored in the ClientSession.  It would hit the keycloak
>>> database as it is not cached yet.  This lookup loads the cache for 
>>> the user.
>>>
>>> Getting in-memory zero-import to work is even more tricky. The issue is
>>> that ClientSession and UserSession need to lookup clients by id.  If 
>>> the
>>> user is not in cache, then the cache needs to lookup the user by id
>>> within storage.  This lookup also needs to happen if a write operation
>>> is performed on a cache user (getDelegateForUpdate()).  So, Keycloak
>>> needs to know that that ID is not in local storage and must be 
>>> looked up
>>> from a fed provider.  The ID must be formed so that the provider fed
>>> provider can resolve the lookup.  I could use a URI for the ID i.e.
>>>
>>> fed:{providerId}:{login-name}
>>>
>>> The problem with this is that this id would need to be larger than 36
>>> characters which is the current column size for UserEntity.id and any
>>> other table that references users.   I could possibly do:
>>>
>>> fed:{providerAlias}:{login-name}
>>>
>>> But its quite possible that combination would be larger than 36
>>> characters.  We could also just shrink it to:
>>>
>>> fed:{login-name}
>>>
>>> But then we would have to iterate over every federation provider to 
>>> find
>>> and load the user.
>>>
>>> So in summary:
>>> * IDs need to expand from 36 characters to something larger. (255
>>> maybe).  Don't some DBs have constraints on string primary key 
>>> size?  DB
>>> scripts could possibly be
>>> * CachedUserProvider and UserFederationManager interfaces would need to
>>> be refactored
>>> * I don't think UserFederationProvider interface would need to change.
>>> But users would have to code for in-memory rather than throwing a 
>>> switch
>>> to just turn it on.
>>>
>>>
>>>
>>
>