Hawkular APM 0.11.0.Final now available
by Gary Brown
Hi
The Hawkular APM team are pleased to announce the release of version 0.11.0.Final.
The release details, including distributions, can be found here: https://github.com/hawkular/hawkular-apm/releases/tag/0.11.0.Final
The release includes:
* Improvements in the UI for displaying service dependency and trace instance information
* Zipkin integration now includes Kafka support (with json and thrift encoded data)
* Initial implementation of a Java opentracing provider
* Integration with Hawkular Alerts to trigger alerts based on trace completion events
Blogs and videos will follow in the next couple of days to demonstrate these capabilities.
Regards
Gary
9 years, 4 months
MiQ log/middleware.log
by mike thompson
So currently, we aren’t really logging anything to this middleware.log (as far as I can tell). Should we be? What is our policy around using this log (versus evm.log)?
This may be more important once we are in CFME and have to debug some customer issues.
Should all of our logging goto this log? Should some? If so what?
Sorry, just a bit confused about the purpose of this log since it shows empty for me.
— Mike
9 years, 4 months
Inventory and postgres ?
by Heiko W.Rupp
Lukas,
I suspect we have an issue in inventory (or two)
- when I run inventory as is with h2db, it works, but may or may not
contribute (a lot) to the growth of the heap as seen in the
"Hawkular-services and memory" thread.
- When I try to run against postgres 9.5, the WildFly of the platform
never is available inside inventory. I see tables in postgres, but the
WF never shows. Also later on I got below error
I am also a bit puzzled, that using postgres starts of a c3p0 connection
pool instead of using what is already present in WildFly
hawkular_1 | 20:11:37,309 WARN [org.hawkular.inventory.rest]
(default task-26) RestEasy exception, :
java.lang.IllegalArgumentException: A metric type with path
'/t;hawkular/f;f38c6e77-6ee0-47da-a80a-bdac9a249457/mt;Singleton%20EJB%20Metrics~Wait%20Time'
not found relative to
'/t;hawkular/f;f38c6e77-6ee0-47da-a80a-bdac9a249457/r;Local~~/r;Local~%2Fdeployment%3Dhawkular-metrics.ear/r;Local~%2Fdeployment%3Dhawkular-metrics.ear%2Fsubdeployment%3Dhawkular-alerts.war/r;Local~%2Fdeployment%3Dhawkular-metrics.ear%2Fsubdeployment%3Dhawkular-alerts.war%2Fsubsystem%3Dejb3%2Fsingleton-bean%3DPartitionManagerImpl'.
hawkular_1 | at
org.hawkular.inventory.base.BaseMetrics$ReadWrite.wireUpNewEntity(BaseMetrics.java:79)
hawkular_1 | at
org.hawkular.inventory.base.BaseMetrics$ReadWrite.wireUpNewEntity(BaseMetrics.java:51)
hawkular_1 | at
org.hawkular.inventory.base.Mutator.doCreate(Mutator.java:168)
hawkular_1 | at
org.hawkular.inventory.base.Mutator.lambda$doCreate$95(Mutator.java:81)
hawkular_1 | at
org.hawkular.inventory.base.TransactionPayload$Committing.lambda$committing$44(TransactionPayload.java:34)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.lambda$inCommittableTxWithNotifications$94(Traversal.java:119)
hawkular_1 | at
org.hawkular.inventory.base.Util.onFailureRetry(Util.java:110)
hawkular_1 | at
org.hawkular.inventory.base.Util.inCommittableTx(Util.java:81)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.inCommittableTxWithNotifications(Traversal.java:118)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.inTxWithNotifications(Traversal.java:91)
hawkular_1 | at
org.hawkular.inventory.base.Mutator.doCreate(Mutator.java:81)
hawkular_1 | at
org.hawkular.inventory.base.BaseMetrics$ReadWrite.create(BaseMetrics.java:122)
hawkular_1 | at
org.hawkular.inventory.base.BaseMetrics$ReadWrite.create(BaseMetrics.java:51)
hawkular_1 | at
org.hawkular.inventory.api.WriteInterface.create(WriteInterface.java:60)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher$1.visitMetric(SingleSyncedFetcher.java:313)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher$1.visitMetric(SingleSyncedFetcher.java:305)
hawkular_1 | at
org.hawkular.inventory.paths.ElementTypeVisitor.accept(ElementTypeVisitor.java:36)
hawkular_1 | at
org.hawkular.inventory.paths.Path$Segment.accept(Path.java:648)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:305)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:350)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:350)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:350)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.lambda$syncTrees$121(SingleSyncedFetcher.java:246)
hawkular_1 | at java.lang.Iterable.forEach(Iterable.java:75)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.syncTrees(SingleSyncedFetcher.java:246)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.lambda$synchronize$119(SingleSyncedFetcher.java:123)
hawkular_1 | at
org.hawkular.inventory.base.TransactionPayload$Committing.lambda$committing$44(TransactionPayload.java:34)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.lambda$inCommittableTx$93(Traversal.java:106)
hawkular_1 | at
org.hawkular.inventory.base.Util.onFailureRetry(Util.java:110)
hawkular_1 | at
org.hawkular.inventory.base.Util.inCommittableTx(Util.java:81)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.inCommittableTx(Traversal.java:105)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.inTx(Traversal.java:96)
hawkular_1 | at
org.hawkular.inventory.base.Traversal.inTx(Traversal.java:79)
hawkular_1 | at
org.hawkular.inventory.base.SingleSyncedFetcher.synchronize(SingleSyncedFetcher.java:93)
hawkular_1 | at
org.hawkular.inventory.base.BaseResources$Single.synchronize(BaseResources.java:206)
hawkular_1 | at
org.hawkular.inventory.rest.RestSync.sync(RestSync.java:80)
hawkular_1 | at
org.hawkular.inventory.rest.RestSync$Proxy$_$$_WeldClientProxy.sync(Unknown
Source)
hawkular_1 | at
sun.reflect.GeneratedMethodAccessor108.invoke(Unknown Source)
hawkular_1 | at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hawkular_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hawkular_1 | at
org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139)
hawkular_1 | at
org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
hawkular_1 | at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
hawkular_1 | at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)
hawkular_1 | at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:395)
hawkular_1 | at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
hawkular_1 | at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
hawkular_1 | at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
hawkular_1 | at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
hawkular_1 | at
javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
hawkular_1 | at
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
hawkular_1 | at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
hawkular_1 | at
io.undertow.websockets.jsr.JsrWebSocketFilter.doFilter(JsrWebSocketFilter.java:129)
hawkular_1 | at
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
hawkular_1 | at
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
hawkular_1 | at
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
hawkular_1 | at
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
hawkular_1 | at
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
hawkular_1 | at
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
hawkular_1 | at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
hawkular_1 | at
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
hawkular_1 | at
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
hawkular_1 | at
io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33)
hawkular_1 | at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
hawkular_1 | at
io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51)
hawkular_1 | at
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
hawkular_1 | at
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
hawkular_1 | at
io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56)
hawkular_1 | at
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
hawkular_1 | at
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
hawkular_1 | at
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
hawkular_1 | at
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
hawkular_1 | at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
hawkular_1 | at
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
hawkular_1 | at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
hawkular_1 | at
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
hawkular_1 | at
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
hawkular_1 | at
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
hawkular_1 | at
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
hawkular_1 | at
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
hawkular_1 | at
io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
hawkular_1 | at
io.undertow.server.HttpServerExchun(HttpServerExchange.java:793)
hawkular_1 | at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
hawkular_1 | at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
hawkular_1 | at java.lang.Thread.run(Thread.java:745)
hawkular_1 |
9 years, 4 months
Get rid of Travis?
by Michael Burman
Hi,
I'm proposing we get rid of the Travis for now (at least for metrics)
and stick to something else, such as the old Jenkins. At the moment, it
can take several runs for metrics PRs to finish, since Cassandra stops
responding in the Travis runs every time. Yesterday I started my PR 4
times, last week one PR for 10 times.
22:28:24,629 ERROR [com.datastax.driver.core.ControlConnection]
(cluster1-reconnection-1) [Control connection] Cannot connect to any
host, scheduling retry in 600000 milliseconds
Sometimes it doesn't even get this far and fails with running the
installation script of Cassandra. These errors have made the Travis runs
completely irrelevant, whenever they fail it just makes sense to restart
them without reading logs (we should get automatic script to restart
always when it fails) until they succeed. No errors reported by Travis
are trustworthy.
We need a working CI solution and this isn't it. There's no "community
visibility" if the results have no meaning, using Jenkins at least
provides us with "failed / not failed", Travis doesn't provide even this.
- Micke
9 years, 4 months
Inventory: transient feeds - or how to tackle the pets vs cattle scenario
by Heiko W.Rupp
Hey,
Right now we identify "agents" via their feed-id.
An instrumented wildfly comes online, registers
its feed with the server, sends its resource discovery
and later metrics with the feed id.
Over its lifecycle, the server may be stopped and re-started
several times.
This is great in the classical use case with installations
on tin or VMs.
In container-land especially with systems like Kubernetes,
containers are started once and after they have died for
whatever reason they are not restarted again.
So the id of an individual container is less and less interesting.
The interesting part is the overall app, that contains of many
containers linked together with several of them representing
an individual service of the app.
So basically we would rather need to record the app and other
metadata for identifying individual parts of the app (e.g. the web
servers or the data bases) and then get pointers to individual
stuff.
The feed would not need to survive for too long, but some of
its collected data perhaps. And then e.g. the discovery of resources
in a new container of the exact same type as before should be sort
of a no-op, as we know this already. Could we short-circuit that
by storing the docker-image-hash (or similar) and once we see this
known one abort the discovery?
Another aspect is certainly that we want to keep (some) historic
records of the died container - e.g. some metrics and the point
when it died. Suppose k8s kills a container and spins a new one
up (same image) on a different node, then logically it is a continuation
of the first one, but in a different place (but they have different feed
ids)
Now a more drastic scenario: As orchestration systems like k8s or
Docker-Swarm have their own registries, that can be queried : do we need
a
hawkular-inventory for this at all?
( We still need it for the non-OpenShift/K8s/Docker-Swarm envs )
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham, Michael O'Neill,
Eric Shander
9 years, 4 months
Hawkular Inventory 0.20.0.Final released
by Lukas Krejci
Hi all,
I'm proud to announce that Hawkular Inventory 0.20.0.Final has been released.
Apart from an updated version of Sqlg and slightly reduced distribution size,
the *big* new feature this version brings is the support for keeping history
of changes to inventory. From now on, you are able to retrieve the history of
updates made to an entity, but you can also "look at" an entity, or whole
inventory, as it looked at certain point in time - questions like "what DB was
this application connected to last month?", "What was the list of feeds
reporting on our production environment in March", etc. All that possible
using a single new query parameter to the REST endpoints - "?at=<TIMESTAMP>".
0.20.0.Final doesn't offer the ability to compact or delete the "old" data
from inventory (delete now merely marks the entity in question as deleted from
the specified time onward) - that will be the focus of our very next release.
We wanted to let this release out though so that people can start enjoying the
nostalgic views at the good old days of their inventory ;)
Special thanks go out to Joel Takvorian for reviews, fixes and the work in
upstream Sqlg.
Thanks,
--
Lukas Krejci
9 years, 4 months
[hawkular-apm]How to inspect query string within a HttpRequest
by 埜陆
hi everyone I just want to make the query string as a property within a
httprequest by setting the businessTransaction
e.g. got two httprequest
GET /foo/bar?text=hello
GET /foo/bar?text=world
and want to classify them with query text by evaluateURL
then node will got a property with name text and values hello and
world
after seeing the apm-javax-servlet.json only the 'processHeaders' will
trigger a businessTranaction process when all other 'setdetails' has not
been invoked which will attach the query string to 'node details' , so the
action 'evaluateUrl' will never take effect, and all other actions like
'setContent' or 'setProperty' only findvalues in Headers and
Content, unfortunately they don't hold the query string which only exists
in details by default.
I am new to this, so am I missing something ? is there any way to achieve
what I want without modify the default config? thanks
best regards
9 years, 4 months
RfC: no longer support Ruby 2.0 in Ruby-Gem?
by Heiko W.Rupp
Hey,
ManageIQ is using Ruby 2.2.4+ and Ruby 2.0 is
pretty old now (Jruby 9.1.x is Ruby 2.3+), so
I would like to no longer "support" Ruby 2.0.
For the time being that only means that
we do not test against 2.0 anymore in Travis.
What do people think?
What about 2.1 ?
Heiko
9 years, 5 months
