<div dir="ltr">With regards to decommissioning I don't think it's up to the user store to do it. We should have a separate migration manager or something that takes care of it. The migration of users from one store is a separate piece of logic to the storage of users in the first place. It should be possible to create a re-usable migration manager that can do the job both for our built in LDAP store and a custom relation db store. Especially when you consider things like:<div><br></div><div>* For a migration that happens when users authenticate (LDAP for example) you want to be able to display progress in the admin console</div><div>* For remaining users at some point you want to decide if they should be dropped, imported without password and if a password recovery email should be sent</div><div>* Probably more stuff to make it real nice</div><div><br></div><div>I added <a href="https://issues.jboss.org/browse/KEYCLOAK-3478">https://issues.jboss.org/browse/KEYCLOAK-3478</a> to cover user migration (or decommissioning of a user provider, not sure what's the best name for it). Ideal would be to have it included in 2.3, but I don't think we have the resources to do that.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 24 August 2016 at 11:58, Marek Posolda <span dir="ltr"><<a href="mailto:mposolda@redhat.com" target="_blank">mposolda@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
<div>On 23/08/16 17:58, Bill Burke wrote:<br>
</div>
<blockquote type="cite">
<p><br>
</p>
<br>
<div>On 8/23/16 10:32 AM, Marek Posolda
wrote:<br>
</div>
<blockquote type="cite">
<div>On 23/08/16 15:04, Bill Burke
wrote:<br>
</div>
<blockquote type="cite">
<p><br>
</p>
<br>
<div>On 8/23/16 3:39 AM, Marek Posolda
wrote:<br>
</div>
<blockquote type="cite">
<div>On 19/08/16 15:52, Bill Burke
wrote:<br>
</div>
<blockquote type="cite">
<p><br>
</p>
<br>
<div>On 8/19/16 2:37 AM, Stian
Thorgersen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 18 August 2016 at 20:30,
Bill Burke <span dir="ltr"><<a href="mailto:bburke@redhat.com" target="_blank"></a><a href="mailto:bburke@redhat.com" target="_blank">bburke@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span><br>
On 8/18/16 4:59 AM, Stian Thorgersen wrote:<br>
> Bill,<br>
><br>
> Are you planing to have an option to
allow import of users with the<br>
> new user federation SPI? I'm not
convinced we should completely remove<br>
> this option.<br>
><br>
<br>
</span>The only callback that does not exist in
the new SPI is<br>
validateAndProxy(). With the current federation
SPI, the developer<br>
implements everything themselves for import.
There are no<br>
synchronization APIs/SPIs either.<br>
</blockquote>
<div><br>
</div>
<div>Sounds like we're removing built-in features
around synchronization just to make the user
have to do everything themselves.</div>
</div>
</div>
</div>
</blockquote>
I think you misinterpreted me, The old User Federation
SPI forces the developer to write all the import code
themselves. The old User Federation SPI does not have any
synchronization callbacks, methods or interfaces other
than validateAndProxy(), the logic of which the user has
to write themselves too.<br>
<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <span>>
Some use-cases I could imagine:<br>
><br>
> * Allow users to authenticate even if
LDAP server is down<br>
</span>Our current LDAP provider will not work
if LDAP is down, even with the<br>
import :)<br>
</blockquote>
<div><br>
</div>
<div>Yes, I know. However, the fact that we don't
currently support it doesn't mean we shouldn't
in the future.</div>
</div>
</div>
</div>
</blockquote>
If the user can only be authenticated via LDAP, an offline
mode is not possible. In other words, if LDAP does not
expose the password of a user (so it can be imported),
then offline mode is not possible. It would only be
possible if the user has logged in at least once, then the
validated password could be imported.<br>
</blockquote>
<blockquote type="cite"> <br>
So, do you still think we should support import/offline
mode given all this?<br>
</blockquote>
From some recent discussions I saw, it seems that quite many
people are interested in the "import-and-forget" mode. So
they need to import user from their old legacy store (3rd
party storage or LDAP) but once user is fully in Keycloak
DB, they want to completely forget about the 3rd party
storage and do all operations around this user against
Keycloak DB.<br>
<br>
The credentials/password validation seems to be the most
complicated part around this as you pointed, as the password
needs to be first successfully validated against 3rdparty
storage or LDAP . Then once password is successfully
validated and updated to Keycloak DB, user can be
"forgotten" and unlinked from the federationProvider. I hope
the new SPI has a way to deal with this usecase? Or at least
have a hook, so the people can easily unlink the user by
themselves whenever they want.<br>
<br>
</blockquote>
As I said before, the current SPI does not have any support
for import. It also does not have any SPIs around
synchronization or any synchronization buttons in the admin
console. It is up to the developer to write the code to
import the user. And our current LDAP implmementation is not
"import and forget", you already mentioned password
validation, but there is also validateAndProxy which is called
every time the user is accessed and which hits LDAP every
time. There's also no way to unlink the user. <br>
</blockquote>
Not right now, but it seems that many people consider the
"import-and-forget" as important usecase? You just want to
import the users from 3rd party storage or LDAP, but you need to
do in multiple steps and "wait" until password is successfully
validated for the first time.<br>
<br>
As an example this blogpost from Scott Rossillo <a href="https://tech.smartling.com/migrate-to-keycloak-with-zero-downtime-8dcab9e7cb2c#.1e8sy1o8n" target="_blank"></a><a href="https://tech.smartling.com/migrate-to-keycloak-with-zero-downtime-8dcab9e7cb2c#.1e8sy1o8n" target="_blank">https://tech.smartling.com/<wbr>migrate-to-keycloak-with-zero-<wbr>downtime-8dcab9e7cb2c#.<wbr>1e8sy1o8n</a>,
which AFAIK seemed to have some positive feedback from more
community users.<br>
<br>
I don't know how deeply to go with directly supporting it at SPI
level. However IMO it will be good to have at least same level
like the current UserFederation SPI. So at least at some point
(ie. after successful password validation), the people can
manually unlink the 3rd party provider from the user and migrate
all the data to Keycloak DB and then use it just from Keycloak
DB.<br>
<br>
</blockquote>
Ok, good feedback. You are convincing me. Are we absolutely sure
this is actually a best practice and not an anti-pattern? Seems
scary to be half in and half out. I guess it does make sense if
you need to keep something like LDAP up for legacy apps.<br>
<br>
<br>
Just thinking around this we should have an additional interface
for imports:<br>
<br>
interface UserStorageSynchornization {<br>
<br>
void validate(UserModel). <br>
void synchronize()<br>
void unlink()<br>
<br>
<br>
}<br>
<br>
<br>
<br>
validate is called whenever a user is looked up. Possibly used to
find deleted users and to synchronize updates on both sides on
demand. I want to add cache policies per provider, so maybe
validate is called only when pulled from persistence storage and
not cache.<br>
<br>
I don't think we need different synchronize methods. We should
instead store last sync timestamp and last updated timestamp for
each user and add queries to local storage to find users for a
specific provider that were synced and/or updated after a certain
time. Then the synchronize implementation can make the
assessment on what to synchronize or not. I'd also like to be
able to fire off synchronization in the background and to obtain a
status on it from the admin console. If it fails, how many users
synchronized, and error message, etc.<br>
</blockquote></div></div>
The support for "background" will be nice. That's what we missed
until now.<br>
<br>
If I understand correctly, this will sync between any UserStorage to
any other UserStorage, so it will defacto provider 2-ways sync ? <br><span class="">
<blockquote type="cite"> <br>
unlink() would just be a callback whenever the admin console fires
of an unlink all users event.<br>
</blockquote></span>
Sounds good to have callback for unlink. Still it will be good to
have possibility to unlink individual users at specific moment
(again, the example when you want to unlink user "john" from LDAP
after he successfuly authenticated to LDAP with password "bar" as
you can then immediatelly update the password to Keycloak DB and
hence you don't need LDAP anymore).<br>
<br>
So for example the usecase like:<br>
1) Keycloak configured with LDAP with 1000 users<br>
<br>
2) 600 users authenticated with their passwords during week1, so
they were already unlinked from LDAP as their passwords (And whole
profile) imported to Keycloak DB<br>
<br>
3) After week1, admin triggers the "unlink" event from admin
console. At this point he wants to forcefully unlink remaining 400
users from LDAP and import them to Keycloak DB. He will also need to
reset their password and send them email etc. This all can be
implemented in the "unlink" callback method right?<br>
<br>
Not sure whether to support alternative of step3, like:<br>
3.a) After week1, admin sends email to remaining 400 users like
"Hey, please login in next 7 days. Otherwise your password will be
restarted."<br>
3.b) After week2, the real unlink is done with the password reset of
users, which didn't login in both week1 and week2.<br>
<br>
Not sure if just "unlink" method is sufficient then...<br>
<br>
Overally it seems that the userStorage is super-complicated as
various people have various use-cases and almost everyone has a bit
different requirements and it's almost impossible to properly
support everything. So IMO it's good if SPI has enough
callbacks/extension points, so people can hook their actions and
eventually implement themselves exactly what they want.<br>
<br>
Marek <br>
<blockquote type="cite"> <br>
Bill<br>
</blockquote>
<br>
</div>
</blockquote></div><br></div>