4:29 a.m.
Hi everybody,
We are currently trying to get Infinispan Cache Metrics via JMX out of Keycloak and into
Prometheus. After including the Prometheus JXM Exporter
(https://github.com/prometheus/jmx_exporter) and going through a configuration spree, we
get warnings in the Keycloak log indicating that the attribute
“rebalancingEnabled” is null/not set for the Infinispan Caches userRevisions,
authorizationRevisions, and realmRevisions. The error looks like the following:
07:32:34,276 DEBUG [org.infinispan.jmx.ResourceDMBean] (pool-1-thread-3) Exception while
reading value of attribute rebalancingEnabled:
java.lang.reflect.InvocationTargetException
07:32:34,276 WARN [org.infinispan.jmx.ResourceDMBean] (pool-1-thread-3) ISPN000036: Did
not find attribute rebalancingEnabled
07:32:34,336 WARN [org.infinispan.topology.CacheTopologyControlCommand] (pool-1-thread-3)
ISPN000071: Caught exception when handling command
CacheTopologyControlCommand{cache=authorizationRevisions, type=POLICY_GET_STATUS,
sender=keycloak-mssql-6599fb56cb-vhll5, joinInfo=null, topologyId=0, rebalanceId=0,
currentCH=null, pendingCH=null, availabilityMode=null, actualMembers=null, throwable=null,
viewId=0}: java.lang.NullPointerException
at
org.infinispan.topology.ClusterTopologyManagerImpl.isRebalancingEnabled(ClusterTopologyManagerImpl.java:628)
at
org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:197)
at
org.infinispan.topology.CacheTopologyControlCommand.perform(CacheTopologyControlCommand.java:153)
at
org.infinispan.topology.LocalTopologyManagerImpl.executeOnCoordinator(LocalTopologyManagerImpl.java:606)
at
org.infinispan.topology.LocalTopologyManagerImpl.isCacheRebalancingEnabled(LocalTopologyManagerImpl.java:540)
at
org.infinispan.cache.impl.CacheImpl.isRebalancingEnabled(CacheImpl.java:998)
at sun.reflect.GeneratedMethodAccessor554.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.infinispan.jmx.ResourceDMBean$InvokableSetterBasedMBeanAttributeInfo.invoke(ResourceDMBean.java:394)
at
org.infinispan.jmx.ResourceDMBean.getNamedAttribute(ResourceDMBean.java:298)
at org.infinispan.jmx.ResourceDMBean.getAttributes(ResourceDMBean.java:197)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttributes(DefaultMBeanServerInterceptor.java:709)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttributes(JmxMBeanServer.java:705)
at
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxScraper.scrapeBean(JmxScraper.java:151)
at
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxScraper.doScrape(JmxScraper.java:117)
at
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxCollector.collect(JmxCollector.java:456)
at
io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183)
at
io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216)
at
io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137)
at
io.prometheus.jmx.shaded.io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)
at
io.prometheus.jmx.shaded.io.prometheus.client.exporter.HTTPServer$HTTPMetricHandler.handle(HTTPServer.java:59)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)
at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82)
at
sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)
at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
These caches are created programmatically, as far as I can see so there is no easy way to
change their configuration. Did anybody have a similar problem or can shed some light why
these caches miss the “rebalancingEnabled” attribute?
Thanks and best regards,
Sebastian
Mit freundlichen Grüßen / Best regards
Dr.-Ing. Sebastian Schuster
Engineering and Support (INST/ESY1)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY |
www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
Sebastian.Schuster@bosch-si.com<mailto:Sebastian.Schuster@bosch-si.com>
Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber,
Michael Hahn