[keycloak-user] User Export with Storage Providers inefficient
Hi,
I was doing some research on an issue we encountered with user export
(at boot-time). The export task was running for hours with ~50.000
LDAP-backed Users and eventually crashed. It obviously got slower and
slower with each user bulk.
I observerd that this also happens with local-only users, as soon as an
LDAP provider is configured and enabled.
SQL log output showed, that for each user-select for a given "page"
(limit and offset), all preceding pages are queried first, which
explains the deterioration over time (quadratic complexity).
The responsible Code (KC 4.8.3) is UserStorageManager#query(...). If any
enabled storage providers exist, this method queries all pages up the
requested one.
I then found this explanation of Summer 2016:
http://lists.jboss.org/pipermail/keycloak-dev/2016-June/007448.html
Right now I've implemented something that is pretty inefficient
to keep it backward compatible right now. Basically I iterate all providers from the
beginning until the page desired is identified and filled up. Minimally it is a stop gap
until I get everything working.
... so it seems to be a concession to backwards compatibility, back when
storage federation got refactored.
Can you think of workaround to make user export usable for us?
Do you plan to drop or improve the current pagination behavior at some
point?
Thanks,
best regards,
Mario.