Re: [keycloak-user] Users to group mapping

Hello Shweta, thank you for checking. On Thu, Aug 29, 2019 at 9:52 PM Shetty, Shweta <Shweta.Shetty(a)teradata.com&gt; wrote: This is a known WIP issue <https://issues.jboss.org/browse/KEYCLOAK-8253>;, we are aware of and working on the fix. FWICT looking at the code, it is currently not possible to switch of the mappers sync <https://github.com/keycloak/keycloak/blob/master/federation/ldap/src/main... (use some configuration option, the mappers sync not to be performed), when the users sync is performed. Mappers sync (thus including the groups one) is called always, when user sync is requested (either "*Periodic Full sync*" or "*Periodic Changed users sync*" options are turned on). But what you can do, till the final fix for KEYCLOAK-8253 <https://issues.jboss.org/browse/KEYCLOAK-8253> is available, are the following two options: - Keep "*Periodic Full sync*" or "*Periodic Changed users sync*" options enabled, but decrease the "*Batch Size*" setting / value in the "*Sync Settings*" section, from the default value of 1000 to something smaller (100, 10 or even 1), depending on the login time delay vs group sync time improvement, when using these lowered options. You would need to try / experiment a bit, which value improves the sync time in your environment. *Rationale:* While in general to improve the time / duration it takes to persist X records to the DB backend, it's better to process more records on one transaction, from the KEYCLOAK-8253 testing, performed so far, the default (1k records per one transaction setting) doesn't provide expected performance gain, when the DB backend doesn't run on the same host, or in the same network. In such a case, attempt to process 1k records per one transaction often results the transaction to fail to be processed as a whole. Thus can't be committed, needs to be rolled back, and retried again. This is, what seems to be (at least from the testing performed so far), causing the delay. Therefore lowering the count of records, being processed within one transaction, it to succeed, could help. Of course, to get the proper value for the setting, most suitable for your environment, it might require some testing / experiment with the setting, not to overwhelm the DB backend with too many transaction requests (which would have negative impact on performance again), in the case the batch size is too low [1]. - As an another workaround to try, till the final fix for KEYCLOAK-8253 is available, you can try to disable both the "*Periodic Full sync*" or "*Periodic Changed users sync*" settings altogether (even LDAP users wouldn't be synced regularly in this case), during the working hours, when delayed time, needed to login to Keycloak is inconvenient. And postpone the LDAP groups sync (and also the mappers sync) to be performed in a non-automated, asynchronous way (e.g. during non-working hours, once per day). This asynchronous sync can be achieved: - Either by 'Sync all LDAP groups to Keycloak' in the admin UI (not convenient), - Or by issuing a Keycloak Admin REST API <https://www.keycloak.org/docs-api/7.0/rest-api/index.html> sync request asynchronously, see: - *Trigger sync of users Action can be "triggerFullSync" or "triggerChangedUsersSync"* POST /{realm}/user-storage/{id}/sync - *Trigger sync of mapper data related to ldap mapper (roles, groups, …) direction is "fedToKeycloak" or "keycloakToFed" * POST /{realm}/user-storage/{parentId}/mappers/{id}/sync methods specification for further details. Regards, Jan -- [1] From what I can see, this is the reason, why the currently proposed patch for KEYCLOAK-8253 isn't sufficient yet. It improves the group sync time when both the Keycloak server & the DB backend container image are running on the same host. But since the change modifies the groups sync workflow, to process even groups in 1k records per one transaction manner (not only users, but also groups are synced this way), it's making the situation even worse in the scenario, when the DB backend isn't locally nearby. So it needs to be refactored yet & is WIP effort.