[keycloak-user] User Export with Storage Providers inefficient

[Mario Imber]

Hi,

are there any news on this one?

Should I rather create a Jira task for this user export issue?

Thanks,
regards,
Mario.


Am 27.05.2019 um 10:04 schrieb mario at imber.wien:
> Hi,
> 
> I was doing some research on an issue we encountered with user export
> (at boot-time). The export task was running for hours with ~50.000
> LDAP-backed Users and eventually crashed. It obviously got slower and
> slower with each user bulk.
> 
> I observerd that this also happens with local-only users, as soon as an
> LDAP provider is configured and enabled.
> 
> SQL log output showed, that for each user-select for a given "page"
> (limit and offset), all preceding pages are queried first, which
> explains the deterioration over time (quadratic complexity).
> 
> The responsible Code (KC 4.8.3) is UserStorageManager#query(...). If any
> enabled storage providers exist, this method queries all pages up the
> requested one.
> 
> I then found this explanation of Summer 2016:
> http://lists.jboss.org/pipermail/keycloak-dev/2016-June/007448.html
> 
>> Right now I've implemented something that is pretty inefficient to 
>> keep it backward compatible right now. Basically I iterate all 
>> providers from the beginning until the page desired is identified and 
>> filled up. Minimally it is a stop gap until I get everything working.
> 
> ... so it seems to be a concession to backwards compatibility, back when
> storage federation got refactored.
> 
> Can you think of workaround to make user export usable for us?
> 
> Do you plan to drop or improve the current pagination behavior at some
> point?
> 
> Thanks,
> best regards,
> Mario.
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user