2 p.m.
Hello,
We've built an installation of Keycloak 6.0.1 with cross-datacenter
replication using Infinispan Server 9.4.15.Final.
About three minutes after starting Keycloak, we start getting lots of
sequences of the following three errors:
ISPN000136: Error executing command RemoveExpiredCommand on Cache 'work'
ISPN000136: Error executing command RemoveCommand on Cache 'work'
ISPN000071: Caught exception when handling command SingleXSiteRpcCommand
Please, see the attached log for more detail.
The attached log is the server.log one of the Infinispan servers.
There you can see that around 17:49 the Keycloak server started and a
couple of warnings regarding cache activity are shown. These warnings are
normal as per Keycloak documentation troubleshooting.
At 17:52, the errors start appearing in a non-stop fashion.
This goes on until one of the infinispan sites automatically goes offline
(only one repetition included in the log file, because it was too big for
the mailing list).
The errors are always related to the work cache only.
Note that the log has been anonymized and host, node, site names, IP's,
ports, etc. do not correspond to the real ones. We have made a best effort
not to obfuscate any relevant information, however.
This log corresponds to an Infinispan 9.4.15 and a Keycloak 4.8.3, both in
domain mode, with one node per data center each.
No error appears in Keycloak logs, even in debug mode (or we haven't found
it).
To discard possible mistakes in the configuration, we have also tested it
with the exact configuration in:
https://www.keycloak.org/docs/latest/server_installation/index.html#setup
These are the results for serveral combinations of versions, always with
the exact example configuration from the docs:
Keycloak Infinispan Result
======== ======== ======
6.0.1 9.4.15 Error
4.8.3 9.4.15 Error
4.8.3 9.4.0 Error
4.8.3 9.3.6 Works
6.0.1 9.3.6 Does not connect
5.0.0 9.3.6 Does not connect
In the three first setups, the log looks exactly the same as the attached
log.
In the fourth setup, everything works fine.
In the two last setups, Keycloak couldn't connect to infinispan, probably
because it's too old.
Our conclusion is that there is a bug showing up only for Infinispan 9.4.x
and not for Infinispan 9.3.x.
Given that the same version of Keycloak works with Infinispan 9.3 and
doesn't work with Infinispan 9.4, this may be an Infinispan bug, but we
have not been able to reproduce it by directly writing onto the work
cache; we have always needed to start Keycloak to reproduce it.
For this reason, we cannot totally discard a problem in Keycloak which
would show up only with later versions of Infinispan.
Any help will be appreciated. If you need any further information, don't
hesitate to ask.
Thank you very much,
Juan
Juan Antonio Farré Basurte
Arquitectura e Innovación
Avda. de Extremadura s/n
37187, Aldeatejada - SALAMANCA
M: +34 647 35 09 72| T: +34 923 70 17 75
E-mail: jafarre(a)viewnext.com | Web: www.viewnext.com
------------------------------------------------------------------------------------------------------------------------------------
Este mensaje va dirigido únicamente a la(s) persona(s) y/o entidad(es) arriba
relacionada(s).
Puede contener información confidencial o legalmente protegida. Si no es usted el
destinatario
señalado, le rogamos borre del sistema inmediatamente el mensaje y sus copias. Asimismo
le
informamos que cualquier copia, divulgación, distribución o uso de los contenidos está
prohibida.
---------------
This message is addressed only to the person (people) and / or entities listed above. It
may contain confidential or legally protected information. If you are not the recipient
indicated, please delete the message and its copies immediately from the system. We also
inform that any copy, disclosure, distribution or use of the contents is forbidden
---------------------------------------------------------------------------------------------------------------
Viewnext, S.A. Domicilio Social: Avda. de Burgos 8-A 28036 de Madrid.
telf: 913834060, Fax: 913834090. Reg. M. Madrid: Tomo 3238, Libro:0, Folio: 78, Seccion:
8ª, Hoja M-55112, N.I.F.: A-80157746