Hi,

I am currently having issues with very slow export/import of realms with a large number of users (10K). Both operations take ~10 minutes each.

After digging in the KeyCloak code, I've found out that a lot of "flush" is done at the Hibernate/JPA level (at least 4-5 times per user).

Export:

For the export, there are 4 very slow queries in ExportUtils.exportUser() (https://github.com/keycloak/keycloak/blob/7895eb6a3dd7e4abc16dd401066bd08ed8075ffb/services/src/main/java/org/keycloak/exportimport/util/ExportUtils.java):

* Set<FederatedIdentityModel> socialLinks = session.users().getFederatedIdentities(user, realm);

* Set<RoleModel> roles = user.getRoleMappings();

* List<UserConsentModel> consents = user.getConsents();

* for (GroupModel group : user.getGroups()) {

They seem to be caused by Hibernate that forces a flush by default in "getResultList()". As a workaround I told Hibernate to flush only on commit of the transaction through "-Dorg.hibernate.flushMode=COMMIT" and the export time came down to ~20 seconds.

Could there be any issue of changing the flush mode only for the export? Data shouldn't change in the database at this moment and queries shouldn't return stale data.

Import:

For the import it seems that changing the flushMode isn't sufficient. It should likely not be done. However, I found a few places in the keycloak-model-jpa module where entities are created, persisted, then em.flush() and em.detatch() is called right away without the entity being returned (ex: UserAdapter.grantRole() - https://github.com/keycloak/keycloak/blob/master/model/jpa/src/main/java/org/keycloak/models/jpa/UserAdapter.java).

If I remove all those flush/detach calls, the import process goes down to ~50 seconds. What is the reason for flushing every time an entity is created rather than letting JPA/Hibernate do it when necessary?

Thank you,

Gabriel

Gabriel Lavoie
glavoie@gmail.com