[keycloak-dev] jgroups / infinispan exception while doing load tests - causing cluster to stop working

[Bartosz Majsak]

Hi,

we are running series of performance tests for our cluster KeyCloak setup
and we have encountered some pretty low level exception with
jgroups/infinispan.

TL;DR; version

   - we are performing 450 req/sec over the period of 30 minutes
   - we doing simple GET calls - getting access token from
   auth/realms/test/protocol/openid-connect/token
   - at some point inifinispan / jgroups fails and we have to restart the
   whole cluster. The exception seems to be very low level [1]

For the longer version we have captured our results in the Google doc [2]
(access for Red Hat only at the moment)

Any ideas / suggestion how to troubleshoot this problem more than welcome :)

Cheers,
Bartosz.

[1]
https://gist.github.com/hectorj2f/5cbd98fadeba019dc94af08219183640#file-logs_broken_pod-L220
[2]
https://docs.google.com/a/redhat.com/document/d/1gcuQbEL2ZTiVcmy39EFMVV9fZxCaS3Oj6ikKZHk9q18/edit?usp=sharing
​