[jboss-jira] [JBoss JIRA] (WFLY-13188) Exception while exporting metrics during WildFly initialization
Brian Stansberry (Jira)
issues at jboss.org
Tue Mar 3 16:52:00 EST 2020
[ https://issues.redhat.com/browse/WFLY-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Stansberry updated WFLY-13188:
------------------------------------
Description:
When the open tracing system is enabled and some metrics are exposed, if when the WildFly is in the startup process, the Prometheus server calls the /metrics endpoint, WildFly prints a stacktrace for each metrics that is has detected (i.e. more than thousands) with the following message:
*WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available
*
In some cases the error persists, even after the server startup is done and we need to restart the WildFly; but we couldn't find the exact scenario for that.
Here is an example for wildfly_jpa_second_level_cache_size_in_memory metric:
{code}
2020-03-02 09:38:06,739 WARN [management I/O-2] [metrics] Unable to export metric wildfly_jpa_second_level_cache_size_in_memory: java.lang.IllegalStateException: WFLYMETRICS0003: Unable to read attribute second-level-cache-size-in-memory on [
("deployment" => "application.ear"),
("subdeployment" => "application-persistence.jar"),
("subsystem" => "jpa"),
("hibernate-persistence-unit" => "application.ear/application-persistence.jar#app"),
("entity-cache" => "com.app.persistence.entity.Country")
]: "WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available".
at org.wildfly.extension.microprofile.metrics.MetricCollector.readAttributeValue(MetricCollector.java:303)
at org.wildfly.extension.microprofile.metrics.MetricCollector.access$200(MetricCollector.java:70)
at org.wildfly.extension.microprofile.metrics.MetricCollector$2.getValue(MetricCollector.java:174)
at org.wildfly.extension.microprofile.metrics.MetricCollector$2.getValue(MetricCollector.java:171)
at io.smallrye.metrics.exporters.OpenMetricsExporter.createSimpleValueLine(OpenMetricsExporter.java:445)
at io.smallrye.metrics.exporters.OpenMetricsExporter.exposeEntries(OpenMetricsExporter.java:178)
at io.smallrye.metrics.exporters.OpenMetricsExporter.getEntriesForScope(OpenMetricsExporter.java:150)
at io.smallrye.metrics.exporters.OpenMetricsExporter.exportAllScopes(OpenMetricsExporter.java:101)
at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:116)
at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:73)
at org.wildfly.extension.microprofile.metrics.MetricsContextService$1.handleRequest(MetricsContextService.java:81)
at org.jboss.as.domain.http.server.security.RealmReadinessHandler.handleRequest(RealmReadinessHandler.java:51)
at org.jboss.as.domain.http.server.security.ServerErrorReadinessHandler.handleRequest(ServerErrorReadinessHandler.java:35)
at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)
at io.undertow.server.handlers.ChannelUpgradeHandler.handleRequest(ChannelUpgradeHandler.java:211)
at io.undertow.server.handlers.cache.CacheHandler.handleRequest(CacheHandler.java:92)
at io.undertow.server.handlers.error.SimpleErrorPageHandler.handleRequest(SimpleErrorPageHandler.java:78)
at io.undertow.server.handlers.CanonicalPathHandler.handleRequest(CanonicalPathHandler.java:49)
at org.jboss.as.domain.http.server.ManagementHttpRequestHandler.handleRequest(ManagementHttpRequestHandler.java:57)
at org.jboss.as.domain.http.server.cors.CorsHttpHandler.handleRequest(CorsHttpHandler.java:75)
at org.jboss.as.domain.http.server.ManagementHttpServer$UpgradeFixHandler.handleRequest(ManagementHttpServer.java:666)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
{code}
was:
When the open tracing system is enabled and some metrics are exposed, if when the WildFly is in the startup process, the Prometheus server calls the /metrics endpoint, WildFly prints a stacktrace for each metrics that is has detected (i.e. more than thousands) with the following message:
*WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available
*
In some cases the error persists, even after the server startup is done and we need to restart the WildFly; but we couldn't find the exact scenario for that.
Here is an example for wildfly_jpa_second_level_cache_size_in_memory metric:
2020-03-02 09:38:06,739 WARN [management I/O-2] [metrics] Unable to export metric wildfly_jpa_second_level_cache_size_in_memory: java.lang.IllegalStateException: WFLYMETRICS0003: Unable to read attribute second-level-cache-size-in-memory on [
("deployment" => "application.ear"),
("subdeployment" => "application-persistence.jar"),
("subsystem" => "jpa"),
("hibernate-persistence-unit" => "application.ear/application-persistence.jar#app"),
("entity-cache" => "com.app.persistence.entity.Country")
]: "WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available".
at org.wildfly.extension.microprofile.metrics.MetricCollector.readAttributeValue(MetricCollector.java:303)
at org.wildfly.extension.microprofile.metrics.MetricCollector.access$200(MetricCollector.java:70)
at org.wildfly.extension.microprofile.metrics.MetricCollector$2.getValue(MetricCollector.java:174)
at org.wildfly.extension.microprofile.metrics.MetricCollector$2.getValue(MetricCollector.java:171)
at io.smallrye.metrics.exporters.OpenMetricsExporter.createSimpleValueLine(OpenMetricsExporter.java:445)
at io.smallrye.metrics.exporters.OpenMetricsExporter.exposeEntries(OpenMetricsExporter.java:178)
at io.smallrye.metrics.exporters.OpenMetricsExporter.getEntriesForScope(OpenMetricsExporter.java:150)
at io.smallrye.metrics.exporters.OpenMetricsExporter.exportAllScopes(OpenMetricsExporter.java:101)
at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:116)
at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:73)
at org.wildfly.extension.microprofile.metrics.MetricsContextService$1.handleRequest(MetricsContextService.java:81)
at org.jboss.as.domain.http.server.security.RealmReadinessHandler.handleRequest(RealmReadinessHandler.java:51)
at org.jboss.as.domain.http.server.security.ServerErrorReadinessHandler.handleRequest(ServerErrorReadinessHandler.java:35)
at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)
at io.undertow.server.handlers.ChannelUpgradeHandler.handleRequest(ChannelUpgradeHandler.java:211)
at io.undertow.server.handlers.cache.CacheHandler.handleRequest(CacheHandler.java:92)
at io.undertow.server.handlers.error.SimpleErrorPageHandler.handleRequest(SimpleErrorPageHandler.java:78)
at io.undertow.server.handlers.CanonicalPathHandler.handleRequest(CanonicalPathHandler.java:49)
at org.jboss.as.domain.http.server.ManagementHttpRequestHandler.handleRequest(ManagementHttpRequestHandler.java:57)
at org.jboss.as.domain.http.server.cors.CorsHttpHandler.handleRequest(CorsHttpHandler.java:75)
at org.jboss.as.domain.http.server.ManagementHttpServer$UpgradeFixHandler.handleRequest(ManagementHttpServer.java:666)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
> Exception while exporting metrics during WildFly initialization
> ---------------------------------------------------------------
>
> Key: WFLY-13188
> URL: https://issues.redhat.com/browse/WFLY-13188
> Project: WildFly
> Issue Type: Bug
> Components: MP Metrics
> Affects Versions: 18.0.1.Final, 19.0.0.Beta2
> Reporter: Ehsan Zaery Moghaddam
> Assignee: Jeff Mesnil
> Priority: Major
> Labels: Microprofile
>
> When the open tracing system is enabled and some metrics are exposed, if when the WildFly is in the startup process, the Prometheus server calls the /metrics endpoint, WildFly prints a stacktrace for each metrics that is has detected (i.e. more than thousands) with the following message:
> *WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available
> *
> In some cases the error persists, even after the server startup is done and we need to restart the WildFly; but we couldn't find the exact scenario for that.
> Here is an example for wildfly_jpa_second_level_cache_size_in_memory metric:
> {code}
> 2020-03-02 09:38:06,739 WARN [management I/O-2] [metrics] Unable to export metric wildfly_jpa_second_level_cache_size_in_memory: java.lang.IllegalStateException: WFLYMETRICS0003: Unable to read attribute second-level-cache-size-in-memory on [
> ("deployment" => "application.ear"),
> ("subdeployment" => "application-persistence.jar"),
> ("subsystem" => "jpa"),
> ("hibernate-persistence-unit" => "application.ear/application-persistence.jar#app"),
> ("entity-cache" => "com.app.persistence.entity.Country")
> ]: "WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available".
> at org.wildfly.extension.microprofile.metrics.MetricCollector.readAttributeValue(MetricCollector.java:303)
> at org.wildfly.extension.microprofile.metrics.MetricCollector.access$200(MetricCollector.java:70)
> at org.wildfly.extension.microprofile.metrics.MetricCollector$2.getValue(MetricCollector.java:174)
> at org.wildfly.extension.microprofile.metrics.MetricCollector$2.getValue(MetricCollector.java:171)
> at io.smallrye.metrics.exporters.OpenMetricsExporter.createSimpleValueLine(OpenMetricsExporter.java:445)
> at io.smallrye.metrics.exporters.OpenMetricsExporter.exposeEntries(OpenMetricsExporter.java:178)
> at io.smallrye.metrics.exporters.OpenMetricsExporter.getEntriesForScope(OpenMetricsExporter.java:150)
> at io.smallrye.metrics.exporters.OpenMetricsExporter.exportAllScopes(OpenMetricsExporter.java:101)
> at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:116)
> at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:73)
> at org.wildfly.extension.microprofile.metrics.MetricsContextService$1.handleRequest(MetricsContextService.java:81)
> at org.jboss.as.domain.http.server.security.RealmReadinessHandler.handleRequest(RealmReadinessHandler.java:51)
> at org.jboss.as.domain.http.server.security.ServerErrorReadinessHandler.handleRequest(ServerErrorReadinessHandler.java:35)
> at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)
> at io.undertow.server.handlers.ChannelUpgradeHandler.handleRequest(ChannelUpgradeHandler.java:211)
> at io.undertow.server.handlers.cache.CacheHandler.handleRequest(CacheHandler.java:92)
> at io.undertow.server.handlers.error.SimpleErrorPageHandler.handleRequest(SimpleErrorPageHandler.java:78)
> at io.undertow.server.handlers.CanonicalPathHandler.handleRequest(CanonicalPathHandler.java:49)
> at org.jboss.as.domain.http.server.ManagementHttpRequestHandler.handleRequest(ManagementHttpRequestHandler.java:57)
> at org.jboss.as.domain.http.server.cors.CorsHttpHandler.handleRequest(CorsHttpHandler.java:75)
> at org.jboss.as.domain.http.server.ManagementHttpServer$UpgradeFixHandler.handleRequest(ManagementHttpServer.java:666)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
> at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
> at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
> at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
More information about the jboss-jira
mailing list