[keycloak-dev] Import users from new User Federation

[Bill Burke]

On 8/23/16 10:32 AM, Marek Posolda wrote:
> On 23/08/16 15:04, Bill Burke wrote:
>>
>>
>>
>> On 8/23/16 3:39 AM, Marek Posolda wrote:
>>> On 19/08/16 15:52, Bill Burke wrote:
>>>>
>>>>
>>>>
>>>> On 8/19/16 2:37 AM, Stian Thorgersen wrote:
>>>>>
>>>>>
>>>>> On 18 August 2016 at 20:30, Bill Burke <bburke at redhat.com> wrote:
>>>>>
>>>>>
>>>>>     On 8/18/16 4:59 AM, Stian Thorgersen wrote:
>>>>>     > Bill,
>>>>>     >
>>>>>     > Are you planing to have an option to allow import of users
>>>>>     with the
>>>>>     > new user federation SPI? I'm not convinced we should
>>>>>     completely remove
>>>>>     > this option.
>>>>>     >
>>>>>
>>>>>     The only callback that does not exist in the new SPI is
>>>>>     validateAndProxy().  With the current federation SPI, the
>>>>>     developer
>>>>>     implements everything themselves for import. There are no
>>>>>     synchronization APIs/SPIs either.
>>>>>
>>>>>
>>>>> Sounds like we're removing built-in features around 
>>>>> synchronization just to make the user have to do everything 
>>>>> themselves.
>>>> I think you misinterpreted me,  The old User Federation SPI forces 
>>>> the developer to write all the import code themselves.  The old 
>>>> User Federation SPI does not have any synchronization callbacks, 
>>>> methods or interfaces other than validateAndProxy(), the logic of 
>>>> which the user has to write themselves too.
>>>>
>>>>
>>>>>     > Some use-cases I could imagine:
>>>>>     >
>>>>>     > * Allow users to authenticate even if LDAP server is down
>>>>>     Our current LDAP provider will not work if LDAP is down, even
>>>>>     with the
>>>>>     import :)
>>>>>
>>>>>
>>>>> Yes, I know. However, the fact that we don't currently support it 
>>>>> doesn't mean we shouldn't in the future.
>>>> If the user can only be authenticated via LDAP, an offline mode is 
>>>> not possible.  In other words, if LDAP does not expose the password 
>>>> of a user (so it can be imported), then offline mode is not 
>>>> possible.  It would only be possible if the user has logged in at 
>>>> least once, then the validated password could be imported.
>>>>
>>>> So, do you still think we should support import/offline mode given 
>>>> all this?
>>> From some recent discussions I saw, it seems that quite many people 
>>> are interested in the "import-and-forget" mode. So they need to 
>>> import user from their old legacy store (3rd party storage or LDAP) 
>>> but once user is fully in Keycloak DB, they want to completely 
>>> forget about the 3rd party storage and do all operations around this 
>>> user against Keycloak DB.
>>>
>>> The credentials/password validation seems to be the most complicated 
>>> part around this as you pointed, as the password needs to be first 
>>> successfully validated against 3rdparty storage or LDAP . Then once 
>>> password is successfully validated and updated to Keycloak DB, user 
>>> can be "forgotten" and unlinked from the federationProvider. I hope 
>>> the new SPI has a way to deal with this usecase? Or at least have a 
>>> hook, so the people can easily unlink the user by themselves 
>>> whenever they want.
>>>
>> As I said  before, the current SPI does not have any support for 
>> import.  It also does not have any SPIs around synchronization or any 
>> synchronization buttons in the admin console.  It is up to the 
>> developer to write the code to import the user.  And our current LDAP 
>> implmementation is not "import and forget", you already mentioned 
>> password validation, but there is also validateAndProxy which is 
>> called every time the user is accessed and which hits LDAP every 
>> time.  There's also no way to unlink the user.
> Not right now, but it seems that many people consider the 
> "import-and-forget" as important usecase? You just want to import the 
> users from 3rd party storage or LDAP, but you need to do in multiple 
> steps and "wait" until password is successfully validated for the 
> first time.
>
> As an example this blogpost from Scott Rossillo 
> https://tech.smartling.com/migrate-to-keycloak-with-zero-downtime-8dcab9e7cb2c#.1e8sy1o8n, 
> which AFAIK seemed to have some positive feedback from more community 
> users.
>
> I don't know how deeply to go with directly supporting it at SPI 
> level. However IMO it will be good to have at least same level like 
> the current UserFederation SPI. So at least at some point (ie. after 
> successful password validation), the people can manually unlink the 
> 3rd party provider from the user and migrate all the data to Keycloak 
> DB and then use it just from Keycloak DB.
>
Ok, good feedback.  You are convincing me.  Are we absolutely sure this 
is actually a best practice and not an anti-pattern?  Seems scary to be 
half in and half out.  I guess it does make sense if you need to keep 
something like LDAP up for legacy apps.


Just thinking around this we should have an additional interface for 
imports:

interface UserStorageSynchornization {

void validate(UserModel).
void synchronize()
void unlink()


}



validate is called whenever a user is looked up.  Possibly used to find 
deleted users and to synchronize updates on both sides on demand.  I 
want to add cache policies per provider, so maybe validate is called 
only when pulled from persistence storage and not cache.

I don't think we need different synchronize methods.  We should instead 
store last sync timestamp and last updated timestamp for each user and 
add queries to local storage to find users for a specific provider that 
were synced and/or updated after a certain time.   Then the synchronize 
implementation can make the assessment on what to synchronize or not.  
I'd also like to be able to fire off synchronization in the background 
and to obtain a status on it from the admin console.  If it fails, how 
many users synchronized, and error message, etc.

unlink() would just be a callback whenever the admin console fires of an 
unlink all users event.

Bill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160823/5ada6fd9/attachment.html