[keycloak-user] Really slow import of large amount of users

[Erdal, Jonatan]

Hi,

We are trying to migrate data from our old login solution to Keycloak, but we are facing issues with really slow imports. We are trying to do the import at server startup, as stated here: https://www.keycloak.org/docs/2.5/server_admin/topics/export-import.html. We are using directory import, and we have 50 users/file. When initiating the import, it is roughly importing 2 files/minute, and this gets slower the more users we have imported. After 2 hrs, we have successfully imported 88 files, 4400 users.

Here is some more details:

·         Number of users: 174 000

·         Number of realms: 1

·         Number of clients: ~5

o   One of the clients is the main client, this has approx. 9400 client roles.

·         Number of groups in realm: ~400

File structure:
Master-realm.json
Master-users-0.json
Kangaroo-realm.json
Kangaroo-users-0.json
Kangaroo-users-1.json
…
Kangaroo-users-3434.json
Kangaroo-users-3435.json

We have also increase the subsystem transaction timeout to 1800 seconds, and at server startup we are also increasing the jboss.as.management.blocking.timeout and setting it to a high number. Apart from users being slow to import, the realm also takes quite some time to import, 10 minutes, I’m guessing this is due to all the client roles we have. In the logs, I cannot see anything that looks super suspicious. At one time, we got the following logs, but then it continues on as before:

2018-03-22 21:29:14,993 WARN  [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac1004f5:701f6ef0:5ab40b08:10 in state  RUN
2018-03-22 21:29:14,993 WARN  [org.hibernate.resource.transaction.backend.jta.internal.synchronization.SynchronizationCallbackCoordinatorTrackingImpl] (Transaction Reaper Worker 0) HHH000451: Transaction afterCompletion called by a background thread; delaying afterCompletion processing until the original thread can handle it. [status=4]
2018-03-22 21:29:15,039 WARN  [org.hibernate.resource.transaction.backend.jta.internal.synchronization.SynchronizationCallbackCoordinatorTrackingImpl] (Transaction Reaper Worker 0) HHH000451: Transaction afterCompletion called by a background thread; delaying afterCompletion processing until the original thread can handle it. [status=4]
[… I have removed about 30 identical entries for readability here …]
2018-03-22 21:29:15,039 WARN  [org.hibernate.resource.transaction.backend.jta.internal.synchronization.SynchronizationCallbackCoordinatorTrackingImpl] (Transaction Reaper Worker 0) HHH000451: Transaction afterCompletion called by a background thread; delaying afterCompletion processing until the original thread can handle it. [status=4]
2018-03-22 21:29:15,039 WARN  [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffffac1004f5:701f6ef0:5ab40b08:10

Does anyone have any idea why it is so slow, is there anything we can do to speed up the process? Please let me know if you need any more information.

Thanks in advance,
//Jonatan



________________________________

Capgemini is a trading name used by the Capgemini Group of companies which includes Capgemini Sverige AB, a company registered in Sweden (number 556092-3053) whose registered office is at Gustavslundsvägen 131 Box 825 – S-161 24 Bromma.
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.