[keycloak-dev] User Federation Provider Cache

[Marek Posolda]

On 23/06/16 16:31, Bill Burke wrote:
> First on pagination.  I was thinking about this a bit.  Do we really 
> think that admins are going to paginate through 1000s of users?  Or 
> will it be more like a couple of hundred at most?  Right now I've 
> implemented something that is pretty inefficient to keep it backward 
> compatible right now.  Basically I iterate all providers from the 
> beginning until the page desired is identified and filled up.  
> Minimally it is a stop gap until I get everything working.
>
> The API will need to change if we want pagination to work efficiently. 
> Invokers on the API will need to keep an index as we need to know the 
> last provider queried and what was the index for the last entry read. 
> As a result, the REST API would have to change too.
+1 to change the REST API. The admin console already works in a way that 
there are arrows for the "next" page and "previous" page right? So if we 
don't have support for "Go directly for users from 1500 to 1600" it 
won't be a big headache though...

It will be also good for LDAP as some LDAP servers have support for 
pagination extension. This is based on the functionality, that you can 
send pagination query to return first 10 users and LDAP will return you 
a "cookie" for going to next page. So just browsing between "next" page 
and "previous" page is easily possible. The most important LDAP vendors 
like MSAD and RHDS have the support for this.

>
> On 6/23/16 8:53 AM, Stian Thorgersen wrote:
>> Some comments on
>> https://github.com/keycloak/keycloak/wiki/2.0-User-Federation-Storage-SPI: 
>>
>>
>> * Different cache policies per provider may be difficult. Wouldn't that
>> require having a Infinispan cache per provider? That wouldn't be very
>> manageable.
>
> I think there is some flexibility here in the Infinispan API.  I know 
> the old api at least allowed you to define zones in the cache.
>
>> * My vote goes to remove UserFederationProvider unless it has no impact
>> on design of the new providers. We shouldn't sacrifice the
>> usability/design of the new SPI for the sake of this.
>
> With what I got so far, I've written it to be completely backward 
> compatible and allow them to co-exist.
>
>> * We should be careful about changing behavior of LDAP provider as it's
>> supported in product
>
> Honestly, we made a very bad decision to not refactor the federation 
> SPI and to force users to import.  It is going to be a really big 
> headache to migrate existing LDAP users to the new LDAP provider.  
> IMO, it is ultra important to stop importing everything.
Maybe I am missing something, but not sure why is it a headache? Is it 
because of the UUID of user is going to change to new format like 
f:<PROVIDER-UUID> , so users may not be referenced from the applications 
with correct UUID? This looks to be a migration issue for all existing 
federation providers, not just LDAP specific?

Marek
>
>> * JTA - should we support proper JTA transactions for consistency?
>>
>
> Yes, but not all federation providers will be able to take advantage 
> of it, i.e. LDAP.
>
> Bill