[keycloak-user] Major Memory Leak/Consumption When Creating Many Users

[Chris Hairfield]

Scratch the results of the graph I posted. I was running the test
incorrectly. I'll post back with the results of the test run properly.

On Wed, Jun 22, 2016 at 12:38 PM Chris Hairfield <chairfield at gmail.com>
wrote:

> Thomas, this test is run with whatever local database Keycloak defaults
> to. We're using Postgres generally, and we will have more information
> pertaining to tests against Postgres soon.
>
> Stian, thanks for the tips. I am currently running a test to ingest about
> 50m users into the default database with the user cache disabled, 8gb mem
> (Xmx and Xms), and parallel GC threads == processor count.
>
> Though my test is young (430k users ingested), I'm noticing memory
> allocation increasing in lockstep with the number of ingested users. Is it
> expected to continue in this fashion, or is Keycloak designed to level off
> in its memory usage?
>
> [image: increasing-heap.png]
>
>
> On Tue, Jun 21, 2016 at 4:29 PM Stian Thorgersen <sthorger at redhat.com>
> wrote:
>
>> Keycloak by default caches users in-memory, by default it will keep up to
>> 10000 entries cached. You can verify that there's no leak by disabling the
>> user cache provider. See
>> http://keycloak.github.io/docs/userguide/keycloak-server/html/server_cache.html#d4e3187
>>
>> If you're planning on having millions of users I suggest you increase the
>> allocated memory for the JVM (512MB which it seems you have is not
>> sufficient).
>>
>> On 22 June 2016 at 00:29, Chris Hairfield <chairfield at gmail.com> wrote:
>>
>>> Hello,
>>>
>>> When testing Keycloak 1.9.8 by ingesting a few million users, we find
>>> that Keycloak leaks memory until it is rendered unresponsive (see graph).
>>> Increasing JVM memory only increases the time it takes to encounter this
>>> issue.
>>>
>>> We have put together a test project here
>>> <https://github.com/anaerobic/keycloak-leakage> and opened an issue here
>>> <https://issues.jboss.org/browse/KEYCLOAK-3146> as we continue to
>>> investigate. As we are relying on Keycloak as a central infrastructural
>>> component, any help would be greatly appreciated.
>>>
>>> We'll update with more information as we find it.
>>>
>>> Thanks,
>>> Chris
>>>
>>> [image: mem-cpu.png]
>>>
>>> _______________________________________________
>>> keycloak-user mailing list
>>> keycloak-user at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160622/164e88a6/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mem-cpu.png
Type: image/png
Size: 100050 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20160622/164e88a6/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: increasing-heap.png
Type: image/png
Size: 11436 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20160622/164e88a6/attachment-0003.png