Hawkular Inventory 0.18.0.Final Released
by Lukas Krejci
Hi all,
I'm glad to announce the release of Hawkular Inventory 0.18.0.Final.
This release brings big improvements in inventory sync along with a couple of
other bugfixes and improvements. Namely:
* Sync now detects changes in all entity data including general-purpose
properties
* Sync no longer requires the root entity to exist prior to its contents and
sub-tree being synced
* It is possible to sync only certain entity types, leaving the rest of a
synced subtree as-is.
* /hawkular/inventory/entity/.../treeHash REST endpoint works correctly
* Bus integration no longer leaves lingering JMS connections behind
* Metric type now contains a "metricDataType" property that will replace the
confusingly named "type" in a future release. The values of the new property
are lower-cased which is in line with what Hawkular Metrics use. The original
property is still in the JSON payload but clients are urged to start using the
new, consistently named, property.
* It is newly possible to filter (and sort) on "metricDataType",
"identityHash", "contentHash" and "syncHash" properties.
Thanks go out to Jiri Kremser and John Mazzitelli for the big help with
reviewing and designing the changes that went to this release.
--
Lukas Krejci
9 years, 8 months
travis failures and srcdep plugin - need inventory release
by John Mazzitelli
tl;dr; Lukas, I need inventory release so I can avoid using srcdep plugin
I have a srcdep dependency in the agent to pull in a specific version of inventory.
It fails to compile on travis. Some errors about generics that looks to be a bug in javac - a newer version seems to fix it. So I changed travis config so it runs on a different machine that gets a different java version.
I then get travis to fail on the enforcer mvn plugin with no details as to why:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-rules) on project hawkular-inventory-parent: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
There are no specific messages above that explain the failure.
So, I can't get travis to pass my PR build with a inventory srcdep dependency. Thus, I need an inventory release.
9 years, 8 months
[inventory] Reading and querying metric type data type
by Lukas Krejci
Hi all,
while working on [1] I came across an incosistency in how the data type of a
metric type (i.e. gauge, availability, counter, ...) are stored and presented.
When you read a metric type using the REST API, you get something like this
back (some fields ommitted for brevity):
{
"path" : "/t;tnt/mt;myMetricType",
"unit" : "NONE",
"type" : "GAUGE",
"collectionInterval" : 0,
"id" : "myMetricType"
}
The metric data type is called "type" here (somewhat confusingly) and contains
the value in upper case.
If you wanted to filter by it, prior to fix to the above mentioned JIRA, you
had to:
/traversal/...;propertyName=__metric_data_type;propertyValue=gauge
Notice 2 things:
* the name of the property is different
* the value is in lower case
Now the question is how to bring order to this mess. Because the ideal fix
will break the format of the data, I'd like to discuss this with the rest of
the team so that we come to a compromise that will amount to the least amount
of work for all the inventory clients.
There are 2 obvious problems here:
1) the mismatch between the property name in the JSON output and the property
name used when querying
2) the mismatch of the letter case in JSON output and queried property value
For 1) I'd like rename BOTH the JSON output and queried property name to
"metricDataType". We can't "type" because its already taken to mean the type
of the entity and we can't use "__metric_data_type" because it's inconsistent
with the rest of the properties.
For 2) I'd like to consolidate on the upper case usage because that's the
default Jackson serialization of the enum.
Both of these (and especially 1)) can break existing clients.
What would be the path of least resistance for your usecase?
[1] https://issues.jboss.org/browse/HWKINVENT-192
Thanks,
--
Lukas Krejci
9 years, 8 months
Hawkular agent for Karaf?
by Thomas Cunningham
Hi,
I've been talking to Heiko about getting the Hawkular agent to work on
Karaf for the Fuse team.
Currently there's number of JON plugins for Fuse that collect metrics and
expose operations through JMX for the components that are being monitored
(Karaf, Camel, SwitchYard, CXF, ActiveMQ, Fabric, etc). There's a few
components that are also shipped on top of EAP (Camel, SwitchYard) that
would need to be monitored as well, but I think I'd like to get the Karaf
portion finished first.
I'd like to create OSGI bundles for the parts of the hawkular-agent that we
need, maybe create a features file for installing the agent, and then
figure out a way to get the agent started so we could collect some data.
Could someone help point me in the right direction? I've got a bunch of
questions ..
How does the agent get started on EAP?
Which parts of hawkular-agent are wildfly-specific? Which parts would I
need for Karaf to get the agent started?
Would it be beneficial to OSGI-enable hawkular-agent by turning the JARs it
creates into OSGI bundles?
Does hawkular-agent have a lot of dependencies?
Thanks for your help in advance - look forward to working with you on this.
Tom
9 years, 8 months
Getting rid of agent/server password in clear
by Heiko W.Rupp
Hey Mazz and Juca,
tl;dr: we need to get rid of clear text passwords in standalone.xml
for the Docker builds I can run (pseudocode):
docker run -e HAWKULAR_USER=jdoe -e HAWKULAR_PASSWORD=password
pilhuhn/hawkular-services
the startup in the image takes care that jdoe is added to the
users.properties file for JAAS
and the agent gets those env-variables as user/password and the agent
can talk to the
server (see also below).
== Agent side
I recall that in the agent installer you have added some way of
'obfuscating' the password.
I wonder if that exists / can be added to the agent proper so that the
password is not
in standalone.xml in clear and I can pass -e
HAWKULAR_PASS_HASH=dhfadfhsdfadsfads
instead of the password and the agent then sends base64(hash(user +
password-hash))
to the server, which does the same with its local data and compares if
the base64
matches.
Remember that docker inspect <container id> lets you see env-variables
"Env": [
"HAWKULAR_BACKEND=remote",
"HAWKULAR_PASSWORD=password",
== Server side
Passing in the password like above to set up the server is equally bad
(perhaps a tiny bit less, as the
server is usually inside a more secured area than the agents). Here I
can in the startup script
easily replace the call to add-user.sh with some "add user + password if
not exists" logic and
the env-variable gets passed in what add-user.sh would compute and add.
9 years, 8 months
Integration of APM into Hawkular Services
by Gary Brown
Hi
Hawkular APM is currently built as a separate distribution independent from other Hawkular components. However in the near future we will want to explore integration with other components, such as Alerts, Metrics and Inventory.
Therefore I wanted to explore the options we have for building an integrated environment, to provide the basis for such integration work, without impacting the more immediate plans for Hawkular Services.
The two possible approaches are:
1) Provide a maven profile as part of the Hawkular Services build, that will include the APM server. The UI could be deployed separately as a war, or possibly integrated into the UI build?
2) As suggested by Juca, the APM distribution could be built upon the hawkular-services distribution.
There are pros/cons with both approaches:
My preference is option (1) as it moves us closer to a fully integrated hawkular-services solution, but relies on a separate build using the profile (not sure if that would result in a separate release distribution).
Option 2 would provide the full distribution as a release, but the downside is the size of the distribution (and its dependencies, such as cassandra), when user only interested in APM. Unclear whether a standalone APM distribution will still be required in the future - at present the website is structured to support this.
Thoughts?
Regards
Gary
9 years, 8 months
[Inventory] What constitutes a "syncable" change of an entity?
by Lukas Krejci
Hi all,
tl;dr: This probably only concerns Mazz and Austin :)
The subject is a little bit cryptic, so let me explain - this deals with
inventory sync and what to consider a change that is worth being synced on an
entity.
Today whether an entity is update during sync depends on whether some of this
"vital" or rather "identifying" properties change. Namely:
Feed: only ID and the hashes of child entities are considered
ResourceType: only ID and hashes of configs and child operation types are
considered
MetricType: id + data type + unit
OperationType: id + hashes of contained configs (return type and param types)
Metric: id
Resource: id + hashes of contained metrics, contained resources, config and
connection config
>From the above, one can see that not all changes to an entity will result in
the change being synchronized during the /sync call, because for example an
addition of a new generic property to a metric doesn't make its identity hash
change.
I start to think this is not precisely what we want to happen during the /sync
operation.
On one hand, I think it is good that we still can claim 2 resources being
identical, because their "structure" is the same, regardless of what the
generic properties on them look like (because anyone can add arbitrary
properties to them). This enables us to do the ../identical/.. magic in
traversals.
On the other hand the recent discussion about attaching an h-metric ID as a
generic property to a metric iff it differs from its id/path in inventory got
me thinking. In the current set up, if agent reported that it changed the h-
metric ID for some metric, the change would not be persisted, because /sync
would see the metric as the same (because changing a generic property doesn't
change the identity hash of the metric).
I can see 3 solutions to this:
* formalize the h-metric ID in some kind of dedicated structure in inventory
that would contribute to the identity hash (i.e. similar to the "also-known-
as" map I proposed in the thread about h-metric ID)
* change the way we compute the identity hash and make it consider everything
on an entity to contribute (I'm not sure I like this since it would limit the
usefulness of ../identical/.. traversals).
* compute 2 hashes - 1 for tracking the identity (i.e. the 1 we have today)
and a second one for tracking changes in content (i.e. one that would consider
any change)
Fortunately, none of the above is a huge change. The scaffolding is all there
so any of the approaches would amount to only a couple of days work.
WDYT?
--
Lukas Krejci
9 years, 8 months
Command Gateway - zip file is empty
by Juraci Paixão Kröhling
Team,
I was working on a change to hservices.t.g.r.c when I noticed a set of
exceptions, which might have been happening for days now. Basically,
they seem to be caused by this, for hawkular-command-gateway-war.war:
> Caused by: java.util.zip.ZipException: zip file is empty
This happens for the latest master, and only on that specific
environment: it works fine locally on my laptop and works fine on Docker
for Heiko. I couldn't find a reasonable answer for this.
A consequence is that the command-gateway endpoints are not working on
that environment, as the deployment failed. The agent, for instance,
calls the following URL quite often:
http://hawkular02:8080/hawkular/command-gateway/feed/ .
Does anyone has any idea on what might be wrong? Did I miss something
from the logs?
Here's a complete server.log:
https://paste.fedoraproject.org/404906/47075585/
The OS is RHEL 7.2, but not sure how relevant this is.
9 years, 8 months
deployments on server group
by Jiri Kremser
Hi,
today's hawkular inventory+WFagent combo have no information about the
deployments of server groups. So my question is what approach should I use,
if we want to visualize the deployments, possibly datasources as members of
a server group?
When using jboss-cli.sh I can see that a server running in a domain mode
has a deployment subsystem under the /server-group=main-server-group and I
can deploy a war into it and check that it's there. But I can't see
anything like that in the inventory neither for the server group resource
[1] nor for the member "domain server" [2].
Do I have to somehow change the configuration of the WF agent subsystem in
host.xml to report those deployments? I assumed installer should do that
when running with --target-config=.../host.xml
btw. I used this cmd for deploying a war:
deploy
/home/jkremser/workspace/ticket-monster/demo/target/ticket-monster.war
--all-server-groups
[1]:
http://localhost:18080/hawkular/inventory/deprecated/feeds/master.Unnamed...
[2]:
http://localhost:18080/hawkular/inventory/deprecated/feeds/master.Unnamed...
jk
9 years, 8 months
H-Services vs. src-deps
by Juraci Paixão Kröhling
Team,
It seems there's a small confusion about src-deps and H-Services.
H-Services is a module that is released every week, given that there's
at least one new commit since the last release.
The src-deps plugin is very helpful for our daily work, as it allows us
to use a given commit as a dependency for our modules, but it's not
appropriate for released artifacts, as it kinda breaks the stability
promise that "a released artifact uses only released artifacts" that is
common on Maven. Besides, I believe there were problems in the past
between src-deps and the maven-release-plugin.
So, avoid sending PRs with src-deps to H-Services. If for some reason
you really need to, switch it to a proper version before Tuesday morning.
- Juca.
9 years, 8 months
