hawkular-accounts integration with websocket stuff in kettle
by John Mazzitelli
Juca,
I think the time is coming close when I need to wrap authentication around the websockets stuff we have in the server. Right now, any client can connect to our websocket server running in kettle. I know you mentioned that KeyCloak doesn't have an official integration with websockets today, but can we make calls to your accounts API directly to do things like logging in?
In our websockets code, I believe we can get access to the headers that were passed in by the UI client when connecting to websockets (javax.websocket.Session.getRequestParameterMap() is what I assume we can use). So a UI or feed could pass in username/password (just like a normal HTTP client would to connect to our metrics or inventory REST API). We would just need to manually take those username/password credentials and ask accounts to verify the credentials and perhaps give us back a UserPrincipal or whatever it is you can give us.
We will eventually need to use this user principal or whatever to logically link that UI client (and the requests it makes to the server) to the responses of those requests (so those responses can be sent back to the proper client)
Ideas?
--John Mazz
8 years, 8 months
[Metrics] Evaluating performance of #groupBy vs #window for stats computation
by Thomas Segismont
Hi,
Recently, we've ported to RxJava the stats computation code, a part not
fully "reactive" yet. The intention was to be able to compute statistics
on raw data for large period of times, with low requirements on memory
(the initial implementation needed all data loaded at once in a list).
If you're interested in the $subject and don't known what stats
computation is, it's an API metrics provides to give the user stats
(min/max/avg/median/95th percentile) on raw data. The stats are given in
buckets (portions of time). As an example, it's the API you use if you
want monthly stats on a year of data, or hourly stats for a day, ... etc.
The implementation makes use of the Observable#groupBy operator. This
allows to determine the bucket each data point belongs to, and then we
#collect the points in each bucket to compute the stats.
https://github.com/hawkular/hawkular-metrics/blob/master/core/metrics-cor...
As we were talking on a related topic (stats for sliding windows), John
wondered if a solution based on Observable#window would perform equally.
So I've implemented this solution and instrumented the code to determine
where the system was spending the execution time. Then I tried both
solutions with different bucket sizes, metric resolutions, numbers of
buckets.
Here are my observations.
#groupBy and #window behave about the same in our case (I've tried with
a few buckets up to a thousand buckets)
Response time increases linearly with the number of data points loaded
from the C*. Response time here means time elapsed between the moment
Wildfly invokes the JAX-RS handler method and we resume the AsyncResponse.
The bottleneck is the data loading and mapping: 95% of execution time.
C* Row to object mapping is nearly 50% of it.
And my conclusions.
When working with non overlapping buckets (not sliding windows),
readability should determine the use of #groupBy or #window, not
performance.
It seems possible to get a better response time. Currently we transform
a ResultSet into an Observable<Row> with a simple call to
Observable#from (ResultSet is an Iterable<Row>). By default, the C*
driver fetches a page of data, and only when it's entirely consumed,
fetches the following one. But ResultSet has a #fetchMoreResults method
which we could use to fetch pages ahead of time (see
http://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/Res...).
Then, while Rx computation threads would spend time on mapping a Row to
a DataPoint<T>, the C* driver could load more Rows.
Attached are:
- a patch of the changes implemented for testing
- some of the results
Regards,
Thomas
8 years, 8 months
MS4 demo / release
by Thomas Heute
So in the end, with the absence of application deployment, what is left
to demo/announce ?
- Access portal features
- Alert settings
Anything else ?
Thomas
8 years, 8 months
bus/agent master will break things temporarily
by John Mazzitelli
As per a conversation betweek heiko, mike, and myself, I'm merging stuff into master branches of bus and agent that will break things temporarily. This is to start getting HAWKULAR-530 in place. Once completely, hopefully we can work on HWKAGENT-5/HAWKULAR-546 working, but that might not be possible for the wed. release.
8 years, 8 months
inventory and agent status
by Jiri Kremser
Hi,
I'll be on PTO whole next week, so I am sending the status what is done and what still needs to be done here:
Inventory 0.3.0 changed the way it deals with escaped characters, we use the @Encoded annotation in resteasy for path params. This is because of the fact one can pass a path to the resources that contains slashes as a resource separators, but the resourceIds can have also slashes in their ids.
This works quite ok, but if the resourceId contains a slash at the end, it doesn't work.
example:
/inventory/test/resources/parent%2res/child%2fres (ok)
/inventory/test/resources/parent%2f (not - ok) the last slash is lost, even though it's part of the id
To make it short, agent can't work with the latest inventory and I prepared 2 scenarios:
1) just fix the escaping and forget about all the new features introduced in inventory/agent:
to make this work just merge https://github.com/hawkular/hawkular-agent/pull/38/ into agent, release a new version of agent
+ merge the https://github.com/hawkular/hawkular/tree/dev/inventory-0.3.0.Final into hawkular master with the new version of agent (currently there is agent@snapshot)
2) on the agent side repo merge the https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-5-resourcec... which contains the logic for inserting the resource configs grabbed from DMR into inventory, has support for the resource hierarchy, fixes the thing from 1) and moves the resource types and metric types in the inventory topology under the feed (this is questionable, because UI has certain issues with it like "give me all the Wildflies" must be translated into "give me all the feeds and for each feed ask for all the WF resources" - it's done here http://git.io/vsRXr, but still).
next step for this scenario to work is to merge the PR on the ui-services http://git.io/vs8Jj and do the appropriate changes in the hawkular/hawkuar ui, I started this here http://git.io/vsRXr all in all UI works, I saw just some errors when going to the details page for the urls, but it's definitely fixable within next week, oh and it depends on my own ui-services version, so once the my PR is merged into ui-services, this (http://git.io/vs04A) can go away
I vote for 2) of course :]
Just a comment to the resource hierarchy:
currently, the agent considers all the resources equal (flat structure) and the hierarchy is captured by the prefixes, like "~/", "~/foo", "~/foo~bar", etc.
The change in the https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-5-resourcec... branch preserves the ids, but adds the real structure among them on the inventory level.
2lkrejci: this is bad wrt resource hierarchy rest api https://github.com/angular/angular.js/issues/1388 (but they say it should be possible somehow using http://git.io/vs02W)
sorry for lack of the structure, it was more a brain dump.
jk
8 years, 8 months
inventory 0.3.0 released
by Jiri Kremser
Hi,
we've released the new Hawkular inventory version.
Also we don't use the .Final .Alpha .IdontKnow 'adjectives'/tags anymore, just plain sem-ver.
It may make sense for the major releases though.
It's the minor release, because there are new features.
What's new?
* Resources can have configs,
* configs can have a structure (validating against json schema).
* Support for dumping the whole graph into json.
* rest endpoint for getting all the children resources of given resource with given resource type
* a lot of bug fixes
Special thanks to ploffay for his contributions.
jk
8 years, 8 months
Timestamped SNAPSHOTs hard to use
by Peter Palaga
Hi *,
I tried to use $subj for the first time today, and I must say $subj did
not meet my expectations. Namely, the timestamps and buildNumbers are
generated at the instant of deployment of the given artifact, which may
lead to distinct timestampts of artifacts of the same group and same
deploy operation.
So, when I want to use the latest timestamped SNAPSHOT of Alerts in
Hawkular, there is no single timestamped version that I could use for
all Alerts artifacts that I manage using the property
${version.org.hawkular.alerts}. To use the latest timestamped version, I
actually can remove the <version.org.hawkular.alerts> property
altogether. I have to put the latest 0.4.0-SNAPSHOT literal to into all
managed alerts artifacts
(https://github.com/hawkular/hawkular/blob/master/pom.xml#L154 and
https://github.com/hawkular/hawkular/blob/master/pom.xml#L161) then run
mvn versions:lock-snapshots -Psnapshots
and voila, it resolves the versions as follows:
<dependency>
<groupId>org.hawkular.alerts</groupId>
<artifactId>hawkular-alerts-actions-email</artifactId>
<version>0.4.0-20150817.111158-17</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.hawkular.alerts</groupId>
<artifactId>hawkular-alerts-rest</artifactId>
<version>0.4.0-20150817.111150-18</version>
<type>war</type>
</dependency>
Here, you can clearly see that the timestamped versions are not the same.
This guarantees the reproducibility of the builds, but I would not call
it practical. Manual maintenance of these scattered version numbers will
be error prone and cumbersome. One can use mvn versions plugin to handle
the versions.
http://www.mojohaus.org/versions-maven-plugin/examples/advancing-dependen...
Can anybody see a way how to make the timestamped snapshots easier to use?
Thanks,
Peter
8 years, 8 months
Keycloak Authorization Tokens
by Artur Dryomov
Hey everybody,
As you might know I’m working on the Hawkular Android Client project [1].
If you tried to run the application you might saw that we are using OAuth
to get access to Hawkular instances because basic auth is too barbaric.
For testing purposes I use the Docker container named jboss/hawkular-aio
[2] with Hawkular 1.0.0 Alpha 3.
Keycloak provides two types of auth tokens. First—a regular one, expires
after 5 minutes by default. Second—a refresh one, which is used to refresh
a regular one, expires after 30 minutes by default. The thing is, it seems
like Keycloak refreshes refresh tokens, i. e. refresh tokens can expire.
All tokens, including the refresh one of course, are refreshed in the
refresh procedure. In practice it means that if you don’t use the
application for an hour, for example, you are forced to relogin, because
you have no ability to refresh tokens. This sounds weird and personally I
would be pissed to relogin basically every time I use the application.
Am I right? Is this a proper behaviour? Can it be avoided without
reconfiguring Keycloak? Who are we in this world? These are the questions
:-)
Artur.
[1]: https://github.com/hawkular/hawkular-android-client
[2]: https://hub.docker.com/r/jboss/hawkular-aio/
8 years, 8 months
Manage cassandra-all in hawkular-parent
by Peter Palaga
Hi *,
ATM, cassandra-all is managed on three places independently, which is
hard to keep in sync:
org.apache.cassandra:cassandra-all occurences:
org.hawkular.metrics:cassandra-seed-provider:2.1.6
org.hawkular.commons:hawkular-commons-embedded-cassandra-service:2.1.6
org.hawkular.alerts:hawkular-alerts-engine:2.1.1
I'd put cassandra-all newest 2.1.* version (2.1.8) to
dependencyManagement in hawkular-parent unless somebody protests loudly.
Note, that we already manage
com.datastax.cassandra:cassandra-driver-core in parent. The driver's
release cycle is independendent from cassandra-all. The newest 2.1.*
version of the driver is 2.1.7.1 and I am going to upgrade to that one
in Parent.
Thanks,
Peter
8 years, 8 months
