[keycloak-dev] in-memory only federated users

[Bill Burke]

On 12/3/2015 9:17 PM, Adam Young wrote:
> On 12/03/2015 10:29 AM, Bill Burke wrote:
>>
>> On 12/2/2015 10:53 PM, Adam Young wrote:
>>>> Getting in-memory zero-import to work is even more tricky.  The issue is
>>>> that ClientSession and UserSession need to lookup clients by id.  If the
>>>> user is not in cache, then the cache needs to lookup the user by id
>>>> within storage.  This lookup also needs to happen if a write operation
>>>> is performed on a cache user (getDelegateForUpdate()).  So, Keycloak
>>>> needs to know that that ID is not in local storage and must be looked up
>>>> from a fed provider.  The ID must be formed so that the provider fed
>>>> provider can resolve the lookup.  I could use a URI for the ID i.e.
>>>>
>>>> fed:{providerId}:{login-name}
>>> I'd recommend this, and increasing the size of the Database column.
>>>
>> We have many deployments already and some have quite large user
>> databases.  Was really concerned that expanding the column size would
>> piss a lot of people off because of the DB migration required.
>
> Migrations are a fact of life.  SQL ALchemy has decent support for it,
> and I've written it at least once in support of JBoss (back in 2000 IIRC).
>
> The trick is to leave the old columns around, but only use the new ones
> for one iteration, so that old scripts don't fall apart horribly.  Its
> not perfect....
>

We use liquibase.  Its not the actual migration code that is a problem 
for us, it is the fact that these guys may have millions of users and 
personally I don't have a lot of experience in data migration scripts 
and how much pain there is take to expand the column and update indexes 
when a column size is changed for millions of records.

You also need to understand that implementation is not what we need 
feedback on.  We're all veterans here.  What we need feedback on is use 
cases and how people are going to use this stuff.  This whole thread 
started because I'm trying to get feedback on whether we shold just wait 
for Keycloak 2.0 to rework the entire user storage SPI or if we can do 
it now with minimal impact to the SPI and existing deployments.

>>
>>
>> We were talking a few weeks ago about ALL of our IDs becoming URIs.
>> roles, groups, realms, clients, etc.  It would give us the flexibility
>> of federating anything we wanted going further.
> This is awesome, but does not need to be stored in the DB as an URL so
> long as you can compose it.
>


URI not URL

fed:{providerId}:{meaningful-user-id}  is a URI.

>
>> For example,
>> currently admins have to define roles and groups in keycloak.  This
>> sucks if they are already defined and controlled someplace else.
>>
>>> Keystoen did this, but then SHA256 hashed it, makuing it a 64 character
>>> string. We found we were OK so long as we stayed under 255, as that was
>>> the limit mysql imposed for the string columns.
>>>
>>>> The problem with this is that this id would need to be larger than 36
>>>> characters which is the current column size for UserEntity.id and any
>>>> other table that references users.   I could possibly do:
>>>>
>>>> fed:{providerAlias}:{login-name}
>>>>
>>>> But its quite possible that combination would be larger than 36
>>>> characters.  We could also just shrink it to:
>>>>
>>>> fed:{login-name}
>>>>
>>>> But then we would have to iterate over every federation provider to find
>>>> and load the user.
>>> It gets trickier:  you don;'t want one federation provider to step on
>>> the login from an other:  ayoung at coke is a different user from
>>> ayoung at pepsi.  Having the Id be something issued by Keycloak + something
>>> that comes from the Federated IdP is necessary.
>>>
>> Not really.  This is no different than a user logging into the
>> user/password screen.  Keycloak allows you to specify the order of
>> federation providers, but doesn't handle conflicts, i.e. one LDAP server
>> has the same username as another.
>
> You need to deal with this.  The rules have changed with Federation, and
> you will have to deconflict.  Two LDAP server in one organization are
> still under the management of a single user. KeyCloaks biggest potential
> for new deployments is Multi IDP, where they are managed by different
> services.  If you don't make the IdPs safe to share the same table
> space, you end up building the logic in to the code, and not being able
> to enforce it in the DB.
>

I think we're talking about two different things.  User Federation for 
us is User Storage Federation.   One logical view of one or more 
identity stores. One login screen where user enters in their username 
and credentials and some logic behind the scenes to locate and discover 
and authenticate that user based on input from Keycloak login screens. 
Can even be a hybrid store where most metadata is stored in something 
like LDAP and additional data stored in Keycloak.

Then we have brokering which is federating external IDPs.  Here, the 
user picks which IDP they want to log into and Keycloak delegates 
authentication to that external IDP.  An example of this is social 
login.  Picking an IDP can even be done by hooking into Keycloak's 
authentication flow SPI, so a user could enter in a email address, and 
keycloak knows to route to an external IDP based on the domain name of 
the email address.

You also need to understand that keycloak isn't just a SAML or OpenID 
Connect bridge.  From the beginning we have strived to be a turnkey 
solution that provides all necessary login screens, forgot password, 
registration, account service management.



>>
>>> I like in memory, but there are many questions to answer, when a user
>>> comes back a second time, what happens with their Id.  I pushed for
>>> ephemeral for Keystone, and we couldn't make it work.
>>>
>> We track sessions, have offline sessions, and track consents.  So an ID
>> cannot be temporary.
> Temporary and ephemeral are two different things.  If you can always
> calculate the ID, and you always get the same ID for a user, you don't
> necessarily need to write it down.
>

See above URI talk...


Maybe you don't understand what we mean by "in-memory".  Currently we 
import some aspects of each user into local storage even if the user 
lives in LDAP.  We did this so that deployments of Keycloak can still 
use LDAP, but get the full features of Keycloak.  There is a small but 
significant set of users that only want to use us as a SAML or OIDC 
bridge so in that case, it doesn't make sense to import anything.



-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com