[keycloak-user] Keycloak OutOfMemoryError

[Jason Spittel]

        Hello,

           We are currently experiencing an OutOfMemoryError / Memory Leak on our Keycloak servers. This occurs intermittently within a span of a few weeks to months between incidents. When it does happen, the entire server is brought down.

           It's a very small load, less than 3000 users, with default settings across the board. One of the keycloak servers is an identity broker, and the other is an IdP that points to the broker (behind the broker is our actual application).

           Looking at JVM logs, the memory is GC'ed regularly with no long term increase, then suddenly, over a period of 5 minutes, spikes to beyond what is allocated to the server (2GB).

           We ran the Eclipse Memory Analyser against the .hprof file and found this as the memory leak suspect:


        default I/O-4
            at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48)
            at java.util.ArrayDeque.doubleCapacity()V (ArrayDeque.java:162)
            at java.util.ArrayDeque.addLast(Ljava/lang/Object;)V (ArrayDeque.java:252)
            at java.util.ArrayDeque.add(Ljava/lang/Object;)Z (ArrayDeque.java:423)
            at org.xnio.nio.WorkerThread.execute(Ljava/lang/Runnable;)V (WorkerThread.java:591)
            at io.undertow.protocols.ssl.SslConduit.runReadListener(Z)V (SslConduit.java:223)
            at io.undertow.protocols.ssl.SslConduit.access$1300(Lio/undertow/protocols/ssl/SslConduit;Z)V (SslConduit.java:63)
            at io.undertow.protocols.ssl.SslConduit$SslReadReadyHandler.readReady()V (SslConduit.java:1081)
            at io.undertow.protocols.ssl.SslConduit$1.run()V (SslConduit.java:229)
            at org.xnio.nio.WorkerThread.safeRun(Ljava/lang/Runnable;)V (WorkerThread.java:580)
            at org.xnio.nio.WorkerThread.run()V (WorkerThread.java:464)


            Which seems related to this bug:

                https://stackoverflow.com/questions/43661909/keycloak-1-9-4-using-custom-federation-running-out-off-memory

            The dev in that situation put Apache in front of keycloak to handle the SSL and seemed to resolve the issue. We'd prefer not to do this. Following this SO post to the mailing list thread:

                http://lists.jboss.org/pipermail/keycloak-user/2016-June/006771.html

            There was some interest in the bug but it was then was abandoned.

            Now, we are running an older version of Keycloak , 3.1.0.Final. But I looked through all the change logs from 3.1.0.Final to 4.5.0.Final as well as all the Jira Issues between those two versions that have to do with SSL, and found no fixes for this issue.

            Is this a problem that is on the radar of the Keycloak devs? Is this the sort of bugfix that would only be in RH SSO?

            Thanks,

            Jason

[cid:8dad4d85-d402-4612-81a1-ded4d2092813]

[cid:ba354506-fb8c-46a0-b587-1430e9afe9a2]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedImage.png
Type: image/png
Size: 52981 bytes
Desc: pastedImage.png
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20181026/af2e0bf7/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedImage.png
Type: image/png
Size: 45477 bytes
Desc: pastedImage.png
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20181026/af2e0bf7/attachment-0003.png