[keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final

[Thomas Darimont]

Hi Sebastian,

thanks for investigating! Unfortunately, I didn't have time yet to look
further into the problem, but I also observed the slow attributes handling.
Btw. we have around 100k users and ~700k attributes. The
Entity-Attribute-Value data model takes it's toll here...

@Stian so far I've only observed this when scrolling through the user
overview in the admin-console, which also triggers
the mentioned spike in heap usage.

Cheers,
Thomas

Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) <
Sebastian.Schuster at bosch-si.com>:

> Btw. we also have 40K user attributes...
>
> Best regards,
> Sebastian
>
> Mit freundlichen Grüßen / Best regards
>
> Dr.-Ing.  Sebastian Schuster
>
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr.
> Stefan Ferber, Michael Hahn
>
>
>
>
> -----Original Message-----
> From: keycloak-dev-bounces at lists.jboss.org <
> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen
> Sent: Mittwoch, 22. August 2018 21:25
> To: Thomas Darimont <thomas.darimont at googlemail.com>
> Cc: keycloak-dev <keycloak-dev at lists.jboss.org>
> Subject: Re: [keycloak-dev] Potential performance regression between
> 3.4.3.Final and 4.3.0.Final
>
> That's quite worrying. Is it limited to browsing users through the admin
> console or are you seeing bad behaviour elsewhere? Same question applies to
> the heap. My best bet here is that it has that something has changed around
> user querying/caching.
>
> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont <
> thomas.darimont at googlemail.com> wrote:
>
> > Hello Keycloak team,
> >
> > has anyone encountered some performance issues after upgrading 3.4.3
> > to 4.x (4.3.0)?
> >
> > Today I noticed a performance regression while preparing an upgrade
> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment.
> >
> > In our test environment, we have around ~100k test users stored in a
> > postgres-backed database. When we started the server with the new
> > Keycloak version, the migration went through, and everything looked
> > fine at first, but when we tried to browse the list of users via the
> > admin-console, we noticed that the CPU and memory consumption of the
> > server increased significantly, up to a point where Keycloak crashed
> with an OOME.
> >
> > All previous Keycloak versions including 3.4.3 were very modest with
> > their memory requirements and quite happy with ~1g heap.
> > However, that seems to have changed in Keycloak 4.3.0 - there we
> > needed at least 4g to prevent Keycloak from crashing with an OOME.
> > Furthermore, we noticed that the response times for browsing the
> > paginated user view increased significantly as well:
> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In
> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation
> > takes
> > ~7 seconds for a test realm with just 10k users.
> >
> > In the test realm with 100k users, the time to load a single page in
> > the users listing was 66 seconds for version 4.3.0, on average -
> > compared to quite stable 80ms in 3.4.3.
> >
> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5
> > seconds for 100k users, so I assume the processing logic in Keycloak
> > is the culprit.
> >
> > The problem of long load-times can be reproduced with the Keycloak
> > docker images and the in-memory database. I also created a small
> > example project that creates some users with just a few attributes in
> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres
> > database to reproduce the problem.
> > https://github.com/thomasdarimont/kc-user-regression-tester
> >
> > Cheers,
> > Thomas
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>