[keycloak-dev] sync/federation requirements/ideas

[Bill Burke]

On 7/10/2014 3:01 PM, Marek Posolda wrote:
>> The problem with this approach is that new users don't get imported
>> into Keycloak until the sync gets invoked.  Syncing more than once per
>> day, or even once per week may not be feasible.  If the external
>> storage does not have a changelog, syncing would involve iterating
>> through each and every user in external storage and syncing it with
>> the keycloak database.  A full sync could take hours.
> Yes, but how to do it differently?
> (a) For methods like: UserProvider.getUsers(), UserProvider.searchFor***
> we can either:
> 1) retrieve users just from "local" store
> 2) Federate users and merge them from both "local" and "external" store.
>
> Federation approach (2) has quite bad performance issues (especially
> with pagination+sorting), so I would prefer (1). But this really
> requires full sync from external store into KC IMO.
>
>
> (b) Then we have methods for retrieve single user:
> getUserBy*** --- here we can try to retrieve user from 'local' store and
> fallback to 'external'. In that case, user will be synced on demand to
> KC database (similar approach like AuthenticationProvider), but then we
> have the issue that just those users, who were retrieved from
> externalStore will be available in KC.
>
> I think that we can either:
> (1) Federate users from both stores
> (2) Temporarily allow that just some users from LDAP are available in KC
> database (those who were authenticated or for which getUserBy*** was
> called)
> (3) Do full sync
>
> I don't know if you see some other solution, but I would say that if we
> want to avoid (1) and (2) we really need to do full sync. So I would
> imagine that if someone configure externalStore, he will also need to
> trigger full sync to KC to have all users added to Keycloak.


Really depends on the UserProvider implementation IMO.  The UserProvider 
doesn't have to import the users on search/getAll operations. The 
UserModels returned could be proxies.  Probably harder than I'm stating, 
but its quite possible its not.


> Personally I would prefer to always do sync from KC immediatelly as
> event queue doesn't always work.

+1.  We'll probably eventually have to have an event queue, but I'd like 
to avoid it.

>
> But fact is that if externalProvider doesn't support changelog, then how
> to do sync without something like "getAllUsers" (or "syncAllUsers")? I
> know that this is quite overhead, but I really don't see how to do
> "partial" sync from 3rd party provider if it doesn't support changelogs...
>


How do we do external store->Keycloak sync?  *WE* don't.  The user does! 
:)  Again  Have a very simple ChronJob that can be scheduled.  I don't 
think there's any detailed SPI we can write that will solve all user needs.

> Marek
>>
>>
>> IMO it would be something like this:
>>
>> public interface ChronJob {
>>
>>    void invoke(KeycloakSessionFactory factory);
>> }
>>

The ChronJob has full access to local keycloak storage and can begin and 
end transactions as needed.  We can probably provide one out of the box 
for LDAP based on simple mapping metadata that can be configured, but 
anything more complex has to be written by the user.


-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com