[keycloak-dev] sync/federation requirements/ideas

[Bill Burke]

On 7/10/2014 7:41 AM, Marek Posolda wrote:
>
> That's why I would prefer "sync" among "federation" approach.

Which is exactly what I'm proposing.  Except it allows for syncs/imports 
into Keycloak on demand.  You already use AuthenticationProvider in this 
manner!

My impression of a sync model is that Keycloak works with user data in 
local Keycloak storage.  User metadata is imported from external storage 
(either on demand or at startp and periodically synced).  Keycloak can 
manage/modify/augment this imported user data locally.  The changelog is 
obtained from the Sync API and updates sent back to the external storage.


> In
> shortcut with "sync" approach will be users (and eventually their role
> mappings) synced from external store into Keycloak and most of the
> UserProvider methods would deal just with "local" provider. Only
> exception are authentication related methods, and also "update" methods,
> which will need to update local user and also "external" users. But
> SyncProvider won't need to implement methods like getUsers(),
> searchFor**, getUserBy** etc as all users will be available locally.
>

The problem with this approach is that new users don't get imported into 
Keycloak until the sync gets invoked.  Syncing more than once per day, 
or even once per week may not be feasible.  If the external storage does 
not have a changelog, syncing would involve iterating through each and 
every user in external storage and syncing it with the keycloak 
database.  A full sync could take hours.


>
>
>
> In details, I can imagine to have things working this way:
>
> * There will be Sync SPI configured per realm, which will allow to sync
> users from "external store" to Keycloak.
>

+1

> * It will be possible to configure when to sync users. For example it
> can be full sync from LDAP at server startup or triggered from admin
> console. Then some periodic sync (triggered for example once per day).
>

+1

> * I am not sure if we need full sync from Keycloak to "external
> provider" but probably not. Once user, role or user credential will be
> updated in Keycloak, it will be also immediatelly synced to external
> provider (if provider is not read-only).
>

If you don't implement through the UserProvider/UserModel interface then 
you would need an Event model for which the SyncProvider listens to so 
that it can sync the user on demand.

> * It depends on SyncProvider implementation, what user data are synced.
> For example in case of LDAP it will be just email+first,lastName and
> eventually role mappings.
>
> * SyncProvider may also support roles + role mappings from external
> store. SyncProvider will be able to import roles into RealmProvider and
> then particular role mappings into UserProvider.
>
> * Users synced from "external provider" will have link to this provider
> similarly like it's now. In case that provider is read-write, then all
> newly created users in Keycloak will be synced into "external provider"
> immediatelly also with the credential (if syncProvider supports
> particular credential)
>
> I would propose the interface like this to handle both
> sync/authentication (not sure if "SyncProvider" is good name, maybe
> rather "ExternalUserProvider"). I can imagine something like this:
>
>
> public interface SyncProvider {
>

I think your SyncProvider interface is very limiting and even unusable. 
  For example, getAllUsers() is completely unfeasible if there is a 
large user database.


IMO it would be something like this:

public interface ChronJob {

    void invoke(KeycloakSessionFactory factory);
}

ChronJob would be scheduled to run at boot time and/or periodically by 
the admin console.  Periodic and boot time syncs would be implemented 
here.  The sync operation needs to have full control of when 
transactions begin and end so updates/creates/deletes can be executed in 
batches.

On demand syncing would be done through an implementation of 
UserProvider as I proposed earlier.  On-demand syncing is:

* Importing a user on demand
* Updating external storage on demand.

We would also have an event changelog that would be implemented as, or 
act like a persistent JMS Topic.  This event topic shouldn't be used to 
implement on-demand sync, but rather for when the external storage is 
read only.:

interface ChangeEvent {
    enum EventType {
        CREATE, UPDATE, DELETE
    }



    long getTimestamp();
    EventType getEventType();
    String getItemId();
    String getItemType(); // UserModel, RoleModel
}

EventListeners could perform the syncs on demand.  Or, a ChronJob could 
be something like a persistence JMS Topic subscriber and replay change 
events.

We could decide that we don't support user updates for read-only stores. 
  Then, in that case, IMO, we don't need a changelog event queue.




-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com