[keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final

[Stian Thorgersen]

Can you open a bug for this in JIRA please and include as much details as
you can.

On Thu, 23 Aug 2018 at 15:45, Thomas Darimont <
thomas.darimont at googlemail.com> wrote:

> Hi Sebastian,
>
> thanks for investigating! Unfortunately, I didn't have time yet to look
> further into the problem, but I also observed the slow attributes handling.
> Btw. we have around 100k users and ~700k attributes. The
> Entity-Attribute-Value data model takes it's toll here...
>
> @Stian so far I've only observed this when scrolling through the user
> overview in the admin-console, which also triggers
> the mentioned spike in heap usage.
>
> Cheers,
> Thomas
>
> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) <
> Sebastian.Schuster at bosch-si.com>:
>
>> Btw. we also have 40K user attributes...
>>
>> Best regards,
>> Sebastian
>>
>> Mit freundlichen Grüßen / Best regards
>>
>> Dr.-Ing.  Sebastian Schuster
>>
>> Engineering and Support (INST/ESY1)
>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
>> GERMANY | www.bosch-si.com
>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
>> Sebastian.Schuster at bosch-si.com
>>
>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr.
>> Stefan Ferber, Michael Hahn
>>
>>
>>
>>
>> -----Original Message-----
>> From: keycloak-dev-bounces at lists.jboss.org <
>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen
>> Sent: Mittwoch, 22. August 2018 21:25
>> To: Thomas Darimont <thomas.darimont at googlemail.com>
>> Cc: keycloak-dev <keycloak-dev at lists.jboss.org>
>> Subject: Re: [keycloak-dev] Potential performance regression between
>> 3.4.3.Final and 4.3.0.Final
>>
>> That's quite worrying. Is it limited to browsing users through the admin
>> console or are you seeing bad behaviour elsewhere? Same question applies to
>> the heap. My best bet here is that it has that something has changed around
>> user querying/caching.
>>
>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont <
>> thomas.darimont at googlemail.com> wrote:
>>
>> > Hello Keycloak team,
>> >
>> > has anyone encountered some performance issues after upgrading 3.4.3
>> > to 4.x (4.3.0)?
>> >
>> > Today I noticed a performance regression while preparing an upgrade
>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment.
>> >
>> > In our test environment, we have around ~100k test users stored in a
>> > postgres-backed database. When we started the server with the new
>> > Keycloak version, the migration went through, and everything looked
>> > fine at first, but when we tried to browse the list of users via the
>> > admin-console, we noticed that the CPU and memory consumption of the
>> > server increased significantly, up to a point where Keycloak crashed
>> with an OOME.
>> >
>> > All previous Keycloak versions including 3.4.3 were very modest with
>> > their memory requirements and quite happy with ~1g heap.
>> > However, that seems to have changed in Keycloak 4.3.0 - there we
>> > needed at least 4g to prevent Keycloak from crashing with an OOME.
>> > Furthermore, we noticed that the response times for browsing the
>> > paginated user view increased significantly as well:
>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In
>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation
>> > takes
>> > ~7 seconds for a test realm with just 10k users.
>> >
>> > In the test realm with 100k users, the time to load a single page in
>> > the users listing was 66 seconds for version 4.3.0, on average -
>> > compared to quite stable 80ms in 3.4.3.
>> >
>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5
>> > seconds for 100k users, so I assume the processing logic in Keycloak
>> > is the culprit.
>> >
>> > The problem of long load-times can be reproduced with the Keycloak
>> > docker images and the in-memory database. I also created a small
>> > example project that creates some users with just a few attributes in
>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres
>> > database to reproduce the problem.
>> > https://github.com/thomasdarimont/kc-user-regression-tester
>> >
>> > Cheers,
>> > Thomas
>> > _______________________________________________
>> > keycloak-dev mailing list
>> > keycloak-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>