7:30 a.m.
The agent in branch hawkular-1275 can now link Prometheus metric family names/labels into
hawkular inventory (at least, it looks like its working :-)
You will see metrics in inventory that look like the below examples - notice the general
props on the metrics now contain Prometheus metric labels - there is one extra property
called "hawkular.metric.family" and that is, obviously, the Prometheus metric
family name. The rest are Prometheus metric labels. We need to change this, I think. We
should have first-class fields on the inventory Metric object for "String
metricFamily" and "Map<String, String> metricLabels". But for now, we
are using general properties.
So for example, to query the "Pool Create Count" metric data from Prometheus for
the Stateless Session EJB resource called "InventoryServiceIspn" you look up its
metric metadata in hawkular inventory, see the following metadata I show below, and then
you know have to query this in Prometheus:
wildfly_ejb_pool_create_count{deployment=hawkular-inventory-service.war,
name=InventoryServiceIspn, feed-id=mazzlap}
We do not have unique metric IDs being stored yet. We should figure out if we really need
them before we store more data in inventory.
Also, the JMX exporter does not actually put the feed-id label on metrics - that will be
the job of the Prometheus server and its scrape definition. But we know it will need to be
there, so inventory will specify it.
{
"name": "Pool Create Count",
"type": "Pool Create Count",
"unit": "NONE",
"properties": {
"feed-id": "mazzlap",
"name": "InventoryServiceIspn",
"type": "stateless-session",
"hawkular.metric.family": "wildfly_ejb_pool_create_count",
"deployment": "hawkular-inventory-service.war"
}
Here's other examples you will see in inventory:
{
"name": "Max Active Sessions",
"type": "Max Active Sessions",
"unit": "NONE",
"properties": {
"feed-id": "mazzlap",
"hawkular.metric.family":
"wildfly_deployment_max_active_sessions",
"deployment": "hawkular-inventory-service.war"
}
{
"name": "Heap Used",
"type": "Heap Used",
"unit": "BYTES",
"properties": {
"area": "heap",
"feed-id": "mazzlap",
"hawkular.metric.family": "jvm_memory_bytes_used"
}