[keycloak-dev] federated pagination Re: federation commited need feedback

[Marek Posolda]

Just to have things a bit more complicated, the LDAP doesn't support 
"classic" pagination model with offset/limit like JPA, but the 
pagination model with cookie/limit (each search returns "cookie", which 
can be used to search for next page). Some details here 
http://www.ietf.org/rfc/rfc2696.txt...

It is not a problem for the Sync usecase though, as sync process will 
have full control and can process all the pages gradually. However it 
looks that it will affect FederationProvider though? I guess "Result" 
object you proposed would need to store cookie in some property as well. 
For picketlink I've added paginationContext to Picketlink IdentityQuery 
to handle this 
https://github.com/picketlink/picketlink/blob/master/modules/idm/api/src/main/java/org/picketlink/idm/query/IdentityQuery.java#L43 
.

Marek

On 25.7.2014 15:43, Bill Burke wrote:
>
>
> On 7/25/2014 3:20 AM, Marek Posolda wrote:
>>> You're right.  The API would have to change to note the provider that
>>> was last used and how many were consumed for that provider.
>>>
>>> class Result {
>>>
>>>    List<UserModel> results;
>>>
>>>    String lastProvider;
>>>    int lastIndex;
>>>
>>> }
>>>
>>> then UserProvider search would need these methods:
>>>
>>> Result search(criteria..., int maxResults);  // start from beginning
>>> Result search(criteria..., String lastProvider, int lastIndex, int
>>> maxResults);
>> Sorry, I still have doubts;-)
>>
>> For example there are 10 users in Keycloak and just 5 of them are mapped
>> to LDAP. In LDAP there are just those 5 users. Then you want to search
>> for page1 with (lastIndex 0, maxResults 10) and you will retrieve those
>> 10 Keycloak Users. Then you want page2, so you call (lastIndex 10,
>> maxResults 10) and now you retrieve those 5 users from LDAP, but those
>> are same users, which were already retrieved on page1.
>>
>
> Solved by searching for local users where federation link is null 
> only?  The side effect is that the federation provider would also have 
> to check the database to make sure the user hasn't already been 
> imported. This could suck as 1 pagination query could turn into 
> MAX_RESULTS local storage queries