[GSoC] Hawkular Android Client
by Artur Dryomov
Hi everyone,
This year I will be working on the Hawkular Android client application as
part of the Google Summer of Code 2015 program.
The application itself will use Hawkular API and AeroGear SDK. In coming
days I’ll research these areas, especially documentation, and will try to
create some sort of architecture and basic design.
Thank you all for this opportunity!
Artur.
1 year, 3 months
Hawkular Metrics Openshift Containers
by Matt Wringe
I have a new subproject in Hawkular Metrics which sets up creating
components for Openshift/Fabric8
(https://github.com/hawkular/hawkular-metrics/pull/200).
There are 3 main parts
Cassandra: creates a custom seed provider to support
ReplicationControllers in Kubernetes, creates a folder/zip archive which
can be used to generate a Docker image. It may make sense to move the
Cassandra parts out to a separate project.
Hawkular Metrics: creates a folder/zip archive which can be used to
generate a Docker image
Kubernetes: pulls everything together into a single kubernetes
application. Can be used to deploy an application zip into fabric8 (via
drag and drop in the web console or via the maven plugin) or deploy all
the components into Openshift via the kubernetes.json configuration file.
The docker images are not created and deployed to a docker registry as
part of the build, it will just create a folder where you can run the
docker build from. None of the maven docker plugins I looked at seemed
to really work properly, so its still a manual process to do the build
(and push to a registry). Its something which needs to be improved.
The Cassandra service currently only supports adding new nodes to a
cluster and not removing them via the ReplicationController. This is due
to the replication factor being set to be 1 by default (which means when
a node is removed, so is the data it contained).
I believe the docker subproject of hawkular metrics is obsolete and can
be removed
(https://github.com/hawkular/hawkular-metrics/tree/master/docker), but
someone please correct me if I am wrong. It's scripts are referring to
the console which no longer exists as part of the project.
- Matt
1 year, 4 months
Tenant Id - Not Part of URL
by Stefan Negrea
Hello Everybody,
I've been working on a PR for the upcoming Hawkular Metrics release that will remove the tenant id from the end-point URLs. The tenant id will be moved to either a header parameter or a query parameter. The query parameter is in place for cases (such as curl) where setting a header is not possible, difficult, or inconvenient.
Here is an example of the change:
Existing URL:
/{tenantId}/gauge/{metricId}/data
New URL:
/gauge/{metricId}/data
Tenant id set via:
1) header - tenantId
2) query parameter - tenantId
There are two exceptions to this rule, /tenants and /db/{tenantid}/series. The /tenants end-point will be changed into something different in the upcoming releases since it is mostly a management type API that does not belong in the same place with the regular metrics endpoint. And /db/{tenantid}/series end-point is needed in this exact format for compatibility with Influxdb compatible services.
Now, to the merits of this change. The tenant id is volatile, can change any time, and changes to it should be expected; but the rest of the URL is fixed. The second issue is that the tenant id is a security concern. So we were limited in design choices since a security concern was leaking as part of the URL.
So removing the tenant id from the URL will give us permanent & consistent addresses for resources (metrics and metric data points). And we will gain a lot of flexibility on the security side. In the future, users could authenticate with a user/pass combo and the backend would transform that into a tenant id to be used on the request. If the same user later decides to use a tenant id to pass along the request, the URL of the resources would not change. Another expectation is that tenant id is not sufficient, it is typically a combo of id + secret; so we would have resorted to a header or query param for the second piece of information (the secret).
This change will give us the flexibility to adjust the security model (the meaning of tenant ids and ways to validate them) without compromising the URL structure. This will help Hawkular Metrics as it gets integrated into more and more projects and products.
Here are the links to the JIRA and the PR for this change:
https://github.com/hawkular/hawkular-metrics/pull/202
https://issues.jboss.org/browse/HWKMETRICS-68
Thank you,
Stefan Negrea
Software Engineer
1 year, 4 months
New and noteworthy in hawkular-parent 25
by Peter Palaga
Hi *,
hawkular-parent 25 brings the following:
* srcdeps-maven-plugin 0.0.5
* meets the promisses falsely done for 0.0.4:
* less console output
* built without tests
* wildfly-maven-plugin 1.1.0.Alpha4
I have sent PRs to all components repos.
Thanks,
Peter
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
1 year, 4 months
[Inventory] Performance of Tinkerpop3 backends
by Lukas Krejci
Hi all,
to move inventory forward, we need to port it to Tinkerpop3 - a new(ish) and
actively maintained version of the Tinkerpop graph API.
Apart from the huge improvement in the API expressiveness and capabilities,
the important thing is that it comes with a variety of backends, 2 of which
are of particular interest to us ATM. The Titan backend (with Titan in version
1.0) and SQL backend (using the sqlg library).
The SQL backend is a much improved (yet still unfinished in terms of
optimizations and some corner case features) version of the toy SQL backend
for Tinkerpop2.
Back in March I ran performance comparisons for SQL/postgres and Titan (0.5.4)
on Tinkerpop2 and concluded that Titan was the best choice then.
After completing a simplistic port of inventory to Tinkerpop3 (not taking
advantage of any new features or opportunities to simplify inventory
codebase), I've run the performance tests again for the 2 new backends - Titan
1.0 and Sqlg (on postgres).
This time the results are not so clear as the last time.
>From the charts [1] you can see that Postgres is actually quite a bit faster
on reads and can better handle concurrent read access while Titan shines in
writes (arguably thanks to Cassandra as its storage).
Of course, I can imagine that the read performance advantage of Postgres would
decrease with the growing amount of data stored (the tests ran with the
inventory size of ~10k entities) but I am quite positive we'd get competitive
read performance from both solutions up to the sizes of inventory we
anticipate (100k-1M entities).
Now the question is whether the insert performance is something we should be
worried about in Postgres too much. IMHO, there should be some room for
improvement in Sqlg and also our move to /sync for agent synchronization would
make this less of a problem (because there would be not that many initial
imports that would create vast amounts of entities).
Nevertheless I currently cannot say who is the "winner" here. Each backend has
its pros and cons:
Titan:
Pros:
- high write throughput
- backed by cassandra
Cons:
- slower reads
- project virtually dead
- complex codebase (self-made fixes unlikely)
Sqlg:
Pros:
- small codebase
- everybody knows SQL
- faster reads
- faster concurrent reads
Cons:
- slow writes
- another backend needed (Postgres)
Therefore my intention here is to go forward with a "proper" port to
Tinkerpop3 with Titan still enabled but focus primarily on Sqlg to see if we
can do anything with the write performance.
IMHO, any choice we make is "workable" as it is even today but we need to
weigh in the productization requirements. For those Sqlg with its small dep
footprint and postgres backend seems preferable to the huge dependency mess of
Titan.
[1] https://dashboards.ly/ua-TtqrpCXcQ3fnjezP5phKhc
--
Lukas Krejci
9 years, 6 months
Identification of WildFly in container in a Kube/Openshift env
by Heiko W.Rupp
Hey,
[ CC to Federico as he may have some ideas from the Kube/OS side ]
Our QE has opened an interesting case:
https://github.com/ManageIQ/manageiq/issues/9556
where I first thought WTF with that title.
But then when reading further it got more interesting.
Basically what happens is that especially in environments like
Kube/Openshift,
individual containers/appservers are Kettle and not Pets: one goes down,
gets
killed, you start a new one somewhere else.
Now the interesting question for us are (first purely on the Hawkular
side):
- how can we detect that such a container is down and will never come up
with that id again (-> we need to clean it up in inventory)
- can we learn that for a killed container A, a freshly started
container A' is
the replacement to e.g. continue with performance monitoring of the app
or to re-associate relationships with other items in inventory-
(Is that even something we want - again that is Kettle and not Pets
anymore)
- Could eap+embedded agent perhaps store some token in Kube which
is then passed when A' is started so that A' knows it is the new A (e.g.
feed id).
- I guess that would not make much sense anyway, as for an app with
three app servers all would get that same token.
Perhaps we should ignore that use case for now completely and tackle
that differently in the sense that we don't care about 'real' app
servers,
but rather introduce the concept of a 'virtual' server where we only
know
via Kube that it exists and how many of them for a certain application
(which is identified via some tag in Kube). Those virtual servers
deliver
data, but we don't really try to do anything with them 'personally',
but indirectly via Kube interactions (i.e. map the incoming data to the
app and not to an individual server). We would also not store
the individual server in inventory, so there is no need to clean it
up (again, no pet but kettle).
In fact we could just use the feed-id as kube token (or vice versa).
We still need a way to detect that one of those kettle-as is on Kube
and possibly either disable to re-route some of the lifecycle events
onto Kubernetes (start in any case, stop probably does not matter
if he container dies because the appserver inside stops or if kube
just kills it).
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham, Michael O'Neill,
Eric Shander
9 years, 7 months
Integration of APM into Hawkular Services
by Gary Brown
Hi
Hawkular APM is currently built as a separate distribution independent from other Hawkular components. However in the near future we will want to explore integration with other components, such as Alerts, Metrics and Inventory.
Therefore I wanted to explore the options we have for building an integrated environment, to provide the basis for such integration work, without impacting the more immediate plans for Hawkular Services.
The two possible approaches are:
1) Provide a maven profile as part of the Hawkular Services build, that will include the APM server. The UI could be deployed separately as a war, or possibly integrated into the UI build?
2) As suggested by Juca, the APM distribution could be built upon the hawkular-services distribution.
There are pros/cons with both approaches:
My preference is option (1) as it moves us closer to a fully integrated hawkular-services solution, but relies on a separate build using the profile (not sure if that would result in a separate release distribution).
Option 2 would provide the full distribution as a release, but the downside is the size of the distribution (and its dependencies, such as cassandra), when user only interested in APM. Unclear whether a standalone APM distribution will still be required in the future - at present the website is structured to support this.
Thoughts?
Regards
Gary
9 years, 7 months
[Inventory] What constitutes a "syncable" change of an entity?
by Lukas Krejci
Hi all,
tl;dr: This probably only concerns Mazz and Austin :)
The subject is a little bit cryptic, so let me explain - this deals with
inventory sync and what to consider a change that is worth being synced on an
entity.
Today whether an entity is update during sync depends on whether some of this
"vital" or rather "identifying" properties change. Namely:
Feed: only ID and the hashes of child entities are considered
ResourceType: only ID and hashes of configs and child operation types are
considered
MetricType: id + data type + unit
OperationType: id + hashes of contained configs (return type and param types)
Metric: id
Resource: id + hashes of contained metrics, contained resources, config and
connection config
>From the above, one can see that not all changes to an entity will result in
the change being synchronized during the /sync call, because for example an
addition of a new generic property to a metric doesn't make its identity hash
change.
I start to think this is not precisely what we want to happen during the /sync
operation.
On one hand, I think it is good that we still can claim 2 resources being
identical, because their "structure" is the same, regardless of what the
generic properties on them look like (because anyone can add arbitrary
properties to them). This enables us to do the ../identical/.. magic in
traversals.
On the other hand the recent discussion about attaching an h-metric ID as a
generic property to a metric iff it differs from its id/path in inventory got
me thinking. In the current set up, if agent reported that it changed the h-
metric ID for some metric, the change would not be persisted, because /sync
would see the metric as the same (because changing a generic property doesn't
change the identity hash of the metric).
I can see 3 solutions to this:
* formalize the h-metric ID in some kind of dedicated structure in inventory
that would contribute to the identity hash (i.e. similar to the "also-known-
as" map I proposed in the thread about h-metric ID)
* change the way we compute the identity hash and make it consider everything
on an entity to contribute (I'm not sure I like this since it would limit the
usefulness of ../identical/.. traversals).
* compute 2 hashes - 1 for tracking the identity (i.e. the 1 we have today)
and a second one for tracking changes in content (i.e. one that would consider
any change)
Fortunately, none of the above is a huge change. The scaffolding is all there
so any of the approaches would amount to only a couple of days work.
WDYT?
--
Lukas Krejci
9 years, 7 months
