Tenant Id - Not Part of URL
by Stefan Negrea
Hello Everybody,
I've been working on a PR for the upcoming Hawkular Metrics release that will remove the tenant id from the end-point URLs. The tenant id will be moved to either a header parameter or a query parameter. The query parameter is in place for cases (such as curl) where setting a header is not possible, difficult, or inconvenient.
Here is an example of the change:
Existing URL:
/{tenantId}/gauge/{metricId}/data
New URL:
/gauge/{metricId}/data
Tenant id set via:
1) header - tenantId
2) query parameter - tenantId
There are two exceptions to this rule, /tenants and /db/{tenantid}/series. The /tenants end-point will be changed into something different in the upcoming releases since it is mostly a management type API that does not belong in the same place with the regular metrics endpoint. And /db/{tenantid}/series end-point is needed in this exact format for compatibility with Influxdb compatible services.
Now, to the merits of this change. The tenant id is volatile, can change any time, and changes to it should be expected; but the rest of the URL is fixed. The second issue is that the tenant id is a security concern. So we were limited in design choices since a security concern was leaking as part of the URL.
So removing the tenant id from the URL will give us permanent & consistent addresses for resources (metrics and metric data points). And we will gain a lot of flexibility on the security side. In the future, users could authenticate with a user/pass combo and the backend would transform that into a tenant id to be used on the request. If the same user later decides to use a tenant id to pass along the request, the URL of the resources would not change. Another expectation is that tenant id is not sufficient, it is typically a combo of id + secret; so we would have resorted to a header or query param for the second piece of information (the secret).
This change will give us the flexibility to adjust the security model (the meaning of tenant ids and ways to validate them) without compromising the URL structure. This will help Hawkular Metrics as it gets integrated into more and more projects and products.
Here are the links to the JIRA and the PR for this change:
https://github.com/hawkular/hawkular-metrics/pull/202
https://issues.jboss.org/browse/HWKMETRICS-68
Thank you,
Stefan Negrea
Software Engineer
1 year, 7 months
New and noteworthy in hawkular-parent 25
by Peter Palaga
Hi *,
hawkular-parent 25 brings the following:
* srcdeps-maven-plugin 0.0.5
* meets the promisses falsely done for 0.0.4:
* less console output
* built without tests
* wildfly-maven-plugin 1.1.0.Alpha4
I have sent PRs to all components repos.
Thanks,
Peter
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
1 year, 7 months
Hawkular Client Ruby
by Lucas Ponce
Hello,
These days we have had several discussions about the hawkular client ruby.
Today there are two main uses cases (Hawkular Metrics from OpenShift and Hawkular Services for MiQ provider) and there are some complexity in the side effects of updating the clients related to the backward compatibility.
I have started to collect some notes about this
https://docs.google.com/document/d/17bAPbBJ6_2wdXAAg_XIQRMi0_RQfODyOEmdOs...
I hope it can help to gather all requeriments to have a clear picture of what could be a good decision for future implementations, studying the pros/cons.
Please, feel free to add your comments or thoughts about the topic.
Thanks,
Lucas
10 years
Hawkular Organization - Repositories - Vote
by Stefan Negrea
Hello Everybody,
Based on the emails and discussions from the past few weeks, I created a
community poll to finalize the decision on unmaintained Hawkular
repositories. The current Github org has a significant number of projects
that no longer receive development. For a full list of projects please see
[1], projects below red line are not active.
Please vote on what to do with repos below the red line and future repos
that will no longer be developed by Hawkular:
1) Move to new Hawkular-Archive organization
2) Keep in Hawkular organization marked as "obsolete"
Voting will be open until Monday, May 31st, 7:00 am (US Central).
Doodle Poll - http://doodle.com/poll/vnaqgppvunhaweks
Please contact me if you have any questions about the two options or have
any comments/feedback.
[1]
https://docs.google.com/spreadsheets/d/1IzoBP9glz3GgkcwTRF6cobQCylUaZFLhF...
Thank you,
Stefan Negrea
Software Engineer
10 years
SSL by default
by Juraci Paixão Kröhling
Team,
I just sent a PR for hawkular-services [1] that adds SSL support by
default to the distribution.
I'd like you to take a moment and do a couple of simple tests of your
component against this distribution, specially if you perform REST calls
to a component endpoint.
Apart from the Agent, I'm not aware of any REST calls made by individual
components, but I'd need to be aware of any problems that this change
might cause.
My next step is to change the agent to accept certs on our keystore.
A few comments:
- The HTTP port is not redirecting to HTTPS yet. This might require
changes to the individual component's web.xml , which I'll be adding soon.
- The certificate inside the keystore is a self-signed one. Should we
ship it on the main distribution, with instructions telling users to
replace our certificate with a real one? Or should we *not* ship it?
Related question: are we even allowed to ship such keystores?
- As mentioned in the previous point, the cert is self-signed. So, you
might need to add "-k" to curl to bypass the cert verification.
- Authentication with client cert is not yet available.
1 - https://github.com/hawkular/hawkular-services/pull/2
- Juca.
10 years
Reform Inventory REST API
by Lukas Krejci
Hi everyone,
tl;dr
Inventory's REST API is ambiguous and doesn't reflect the generic structure of
the inventory well. Let's change that before it's too late!
The access patterns in inventory's REST API predate the concept of the
canonical path that we now use extensively throughout the model to uniquely
identify the entities and because of that we're running into various issues
with the REST API. From slight inconveniences to outright breakage due to
ambiguous URLs.
Here I propose a reformed REST API centered around the canonical paths. It
should have the same expressive power as the original REST API but should not
suffer from the ambiguities and should be much more cohesive and "logical".
The only thing that was possible using 1 call in the old API that will require
2 calls in the new API is disassociation of 2 entities (i.e. disassociate a
metric from a resource, etc.). These operations are IMHO rather rare so I am
not too worried about this.
I'd like to know your opinions on the new API:
URIs below are defined in EBNF,
CP stands for canonical path of some entity,
REL stands for a name of some relationship (pre-defined or user defined)
== sync endpoint
URI = "/", "sync", "/", CP;
This is the same as it is currently. There is a parallel thread from the last
week about the evolution of sync that will be addressed, too.
== bulk endpoint
URI = "/", "bulk";
might go away - we have sync doing almost the same thing
== GET URLs
This is a little bit complex but what this does is that it enhances a
canonical path as is currently known with the ability to define filters on
each path progression step.
Basically, this is an attempt to express a graph traversal using an URL.
ANY = ? just a URI path-escaped string representing an entity ID or
relationship name ? ;
FILTER_NAME = "type" | "id" | "name" | "cp" | "identical" | "propertyName" |
"propertyValue" ;
FILTER = FILTER_NAME , [ "=", ANY ] ; (* value is not required for the
"identical" filter *)
PATH_SEGMENT_TYPE = "t" | "e" | "mp" | "f" | "rt" | "mt" | "ot" | "r" | "m" |
"d" ;
PATH_SEGMENT_ID = ANY ;
PATH_STEP = "/", PATH_SEGMENT_TYPE, ";", PATH_SEGMENT_ID, { ";", FILTER } ;
WELL_KNOWN_REL = "contains" | "defines" | "incorporates" | "isParentOf" ;
ANY_REL = ANY ;
DIR_FILTER = ";", ( "in" | "out" | "both" )
REL_FILTER = ( "propertyName" | "propertyValue"), "=", ANY ;
REL_STEP = "/rl;", ( WELL_KNOWN_REL | ANY_REL ), [ DIR_FILTER ], { REL_FILTER
} ;
FILTER_STEP = FILTER, { ";", FILTER } ;
RETURN_TYPE = "" | "treeHash" ;
PATH_END = ( PATH_STEP | FILTER_STEP ), RETURN_TYPE ;
URI = { ( PATH_STEP | FILTER_STEP ), [ REL_STEP ] }, [ PATH_END ];
The "identical" bit is currently not present in the REST API but is in the
Java API. What it'd do here is that it would "widen" the start of the query
from the one entity specified by the CP to all entities that are identical to
it according to the identity hash rules (same id, same significant structure).
This is useful for scenarios like "querying all EAPs". The way this would work
is that you'd have your resource type that you expect defined and a global
level, possibly contained in a metadata pack. You'd then look for resources
that are defined by the resource types identical to yours. Because feeds are
free to (re-)define their resource types, this would match resources from
feeds that have types identical to the global one. Note that there is no
special relationship needed between the types - inventory figures this out
automagically. This way we loosen the requirement for synchronizing the
updates to the types defined by feeds and the user at the cost of "eventually
consistent behavior" once the parties upgrade at their own pace.
=== Examples
==== Return a tenant
/
==== Access Entity By Canonical Path
/t;tenant_id/f;feed_id/rt;resourceType_id
This is actually equivalent to:
/t;tenant_id/rl;contains;out/f;feed_id/rl;contains;out/rt;resourceType_id
which is no longer a canonical path but showcases how we declare "hops" over
specific relationships. "rl;contains;out" translates to "relationship with
name contains in the outgoing direction" and is implicit, if no other "hop"
between entities is specified.
To return the tree hash of the entity instead of the entity itself, one can:
/f;feed_id/r;resource/treeHash
Note that the tenant in the path is optional because it can be deduced from
the authorization details.
==== Accessing Targets of Relationships
/f;feed_id/r;resource_id/rl;incorporates/type=metric
This is equivalent to the current
`/feeds/feed_id/resources/resource_id/metrics`.
/f;feed_id/r;resource_id/rl;isParentOf/
(notice the trailing slash)
"give me all children of resource with id 'resource_id'".
==== Accessing Relationships
/f;feed_id/rl;contains
(notice the lack of trailing slash)
"find all the 'contains' relationships going out of the feed with id
'feed_id'."
To access a single relationship with known id:
/rl;relationship_id
==== More Complex Example
/f;feed_id/type=rt;name=My%20EAP;identical/rl;defines/type=resource/rl;isParentOf/type=resource?recursive=true
"get a feed with id 'feed_id' and find all resource types called "My EAP" that
it contains and all other resource types that are identical to it (them). Then
find all the resources that those resource types define and find all the
resources (recursively) that those resources are parents of."
=== Query Parameters
==== Paging `per_page`, `page`, `sort`, `order`
Paging is very expensive, because it implies fully iterating through the
result set (to be able to sort or determine the total). We may think about
some kind of server-side "live result set" that would hold the results on the
server and be accessed using some kind of token (with a TTL). This is how
neo4j does it and would avoid having to fetch the full result set on each
paged request.
==== `recursive`
This causes the last hop (relationship + entity filter) to be recursively
searched and added to the results. Tinkerpop defines a more generic concept of
"loop" using a label as a marker of the start of the "recursive hop" but I
don't think we need to be that powerful in a REST interface. Advanced users
may want to use the `query` endpoint with the full power of Gremlin query.
== POST URLs
URI = "/", CP, "/", "resource" | "metric" | "resourceType" | ...;
The idea here is that you can create the entities only at their canonical
positions. While the Java API allows for creation after a non-canonical
traversal, I think this would be unnecessarily complicated for the REST API.
The users would pass the familiar blueprint objects to these endpoints as they
do currently.
Examples:
/feed - send a feed blueprint and this will create a new feed
/resourceType - send a resource type blueprint and it will create a new global
resource type
/metricType
...
/f;feed/resourceType - creates a feed-local resource type
== PUT URLs
URI = "/", CP
just send an update object corresponding to the type of entity on the path
== DELETE URLs
URI = "/", CP
deletes the entity on the path
disassociation needs to be 2 steps - find the relationship in question and
then
delete the relationship, e.g.:
GET /f;feed_id/r;resource/rl;defines
DELETE /rl;id-found-in-the-results-from-the-previous-query
== Advanced Querying
URI = "/", "query"
free form, read-only gremlin query for more complex queries (this needs
to wait for the port to Tinkerpop3)
10 years
BTM or not BTM, that is the question
by Gary Brown
Hi all
As discussed in the recent blog (and demo), Hawkular BTM now provides Application Performance Management (APM), Distributed Tracing (DT) and Business Transaction Management (BTM).
The project was initially started to address the third area, hence being called BTM - however BTM requires the collection of the information used to provide APM and DT - and providing those other views on the information is useful to users.
Due to the wider scope, just wondering whether the BTM name is now misleading. If a project component rename was considered a good idea, then now would probably be the best time to do it, before getting close to a stable 1.0 version?
So two questions:
1) Is a component rename a good idea?
2) If so, any suggestions for a name, either some generic term that encapsulates APM, DT and BTM - or alternatively a completely unrelated name (but then it would not follow the pattern used by other Hawkular components).
Regards
Gary
10 years
Metrics: Cassandra Driver timeouts during the build
by Peter Palaga
Hi *,
Short version: doubling the read timeouts [1] for all three uses of
Cassandra Driver solved the problem for me. Is this a good idea? Can
anybody see a better solution to my problem?
Long version:
https://issues.jboss.org/browse/HWKMETRICS-404
As some of you already know from my complaints on IRC, I have not been
able to build Metrics since a couple of weeks.
My builds failed mostly on core-services but sometimes also in
configuration-service and resr-tests-jaxrs, the error always being the same:
com.datastax.driver.core.exceptions.NoHostAvailableException: All
host(s) tried for query failed (tried: /127.0.0.1:9042
(com.datastax.driver.core.exceptions.OperationTimedOutException:
[/127.0.0.1] Timed out waiting for server response))
The whole build log: http://paste.fedoraproject.org/372623/46116551/
This started happening at some point after my PR
https://github.com/hawkular/hawkular-metrics/pull/478 was merged. I do
not say this PR is the cause, this is just an attempt to narrow down the
set of commits that might have introduced the problem.
As nobody else (incl. Travis) complained about anything similar, I was
thinking the issue must be somewhere in my environment. I double checked
I use the right Cassandra version, ulimits, Java version, Maven version,
system updates, but nothing helped.
Then I tried doubling the read timeouts [1] for all three uses of
Cassandra Driver and the build started passing again. Is this a good
idea? Can anybody see a better solution to my problem?
[1] https://github.com/hawkular/hawkular-metrics/pull/510
10 years
Progress Report on Android Client of Hawkular
by Anuj Garg
Hello,
I am Anuj. Final year computer Science graduation student and currently
working on Android Client of Hawkular as my GSoC Project.
It is my first report of what is going on the client recently.
*What have been accomplished till yet*
-> A detail screen for alert have been introduced from where you can change
the state of alert read important detail about it. Read and send comments
on the alert.
-> Previously Android client showed only Open Alerts which have been
changed to something similar server. By it shows {OPEN & ACK} when you
press '+' sign it includes resolved alerts.
-> As more status type of alerts are being displayed so, state of alert is
also added in the list view. For which I need some suggestions if this one
is OK, or I should use symbols instead.
*Video of current work*
https://drive.google.com/file/d/0B8tYKC1-UERHSDVSa1hGQ3VEbzQ/view?usp=sha...
*What to come soon*
-> From detailed alert, jump to trigger to view or alter the trigger.
-> Tabbed pane for list of alerts and associated triggers with that
resources.
-> Detail screen of triggers with enabling, disabling the trigger.
Please suggest if any changes are to be done.
Thanks,
Anuj Garg
10 years
