Tenant Id - Not Part of URL
by Stefan Negrea
Hello Everybody,
I've been working on a PR for the upcoming Hawkular Metrics release that will remove the tenant id from the end-point URLs. The tenant id will be moved to either a header parameter or a query parameter. The query parameter is in place for cases (such as curl) where setting a header is not possible, difficult, or inconvenient.
Here is an example of the change:
Existing URL:
/{tenantId}/gauge/{metricId}/data
New URL:
/gauge/{metricId}/data
Tenant id set via:
1) header - tenantId
2) query parameter - tenantId
There are two exceptions to this rule, /tenants and /db/{tenantid}/series. The /tenants end-point will be changed into something different in the upcoming releases since it is mostly a management type API that does not belong in the same place with the regular metrics endpoint. And /db/{tenantid}/series end-point is needed in this exact format for compatibility with Influxdb compatible services.
Now, to the merits of this change. The tenant id is volatile, can change any time, and changes to it should be expected; but the rest of the URL is fixed. The second issue is that the tenant id is a security concern. So we were limited in design choices since a security concern was leaking as part of the URL.
So removing the tenant id from the URL will give us permanent & consistent addresses for resources (metrics and metric data points). And we will gain a lot of flexibility on the security side. In the future, users could authenticate with a user/pass combo and the backend would transform that into a tenant id to be used on the request. If the same user later decides to use a tenant id to pass along the request, the URL of the resources would not change. Another expectation is that tenant id is not sufficient, it is typically a combo of id + secret; so we would have resorted to a header or query param for the second piece of information (the secret).
This change will give us the flexibility to adjust the security model (the meaning of tenant ids and ways to validate them) without compromising the URL structure. This will help Hawkular Metrics as it gets integrated into more and more projects and products.
Here are the links to the JIRA and the PR for this change:
https://github.com/hawkular/hawkular-metrics/pull/202
https://issues.jboss.org/browse/HWKMETRICS-68
Thank you,
Stefan Negrea
Software Engineer
1 year, 6 months
New and noteworthy in hawkular-parent 25
by Peter Palaga
Hi *,
hawkular-parent 25 brings the following:
* srcdeps-maven-plugin 0.0.5
* meets the promisses falsely done for 0.0.4:
* less console output
* built without tests
* wildfly-maven-plugin 1.1.0.Alpha4
I have sent PRs to all components repos.
Thanks,
Peter
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
1 year, 6 months
What are your Authentication and Authorization needs?
by Juraci Paixão Kröhling
Team,
Due to the changes in requirements for Hawkular, I'm collecting the
needs we have around authentication and authorization.
On another thread, I mentioned that I'm inclined to settle on JAAS, but
before I invest more time on that, I'd like to hear from you on your
needs. Ultimately, securing your component is of my interest, but I
don't have a complete picture on what your needs are.
The current plan is to reduce Accounts 3.x to a minimum. Ideally, we
won't even need Accounts 3.x, if we find out that all we need is JAAS.
As a summary, here's what we have in Accounts that might currently be in
use by other modules:
- WebSocket authentication of messages. Each WebSocket message coming to
the server has an "authentication" property, with credentials or tokens.
The idea is that long-running connections would re-authenticate from
time to time, without closing/opening the connection. The obvious easy
solution here is to force-close a connection on the server side, and do
a reconnect on the client. Ideally, however, we would find a way to
authenticate the user with JAAS API. If that's possible, Accounts can
help and consumers of WebSocket auth (ie: command gateway) won't need to
be changed.
- Multi tenancy: so far, I have not heard any concrete use cases for
multi tenancy other than "it would be nice to have". Remembering that
multi tenancy here is different than storing tenant information. We
could certainly keep storing tenant information, but I don't see a case
where we would need to perform authorization there.
- Token support: this is not, per se, supported by vanilla JAAS. On the
other hand, this can be emulated by having multiple "opaque" accounts on
the JAAS user storage. So, this is not a big deal, except that the token
API will cease existing (ie: any REST calls to /secret-store). This
should affect *only* the UI and the Android client.
- Functional users instead of real users: this is the main change. Users
won't be "real" users anymore (ie: jpkroehling), but technical users
(ie: miq-prod-01). Effectively, this is similar to authentication done
by backend services, such as Cassandra and Postgresql. For us, this
would be 100% handled by JAAS, so, we don't need to worry whether users
are real or functional, but we would recommend using functional users.
- Authorization: we currently have an API for authorization. I don't
think anyone is using that, except perhaps for Inventory. As such, I
would just remove it in favor of JAAS' authorization. This effectively
means that you should be able to annotate "@RolesAllowed()" into your
business methods, and JAAS would ensure that the user belongs to any of
the authorized roles. We would, however, need to define the roles we
need. I'd suggest starting with "read-only" and "read-write" as roles.
- Events from the auth server: components are currently able to register
a listener on a JMS queue or CDI event, to receive Keycloak server
events (user logged in, user registered, ...). This will also cease to
exist. It was implemented as a requirement from Inventory, but I think
it's not being used anymore.
- Feature pack: components building feature packs or consuming Accounts
feature pack would need to use Nest instead. I don't expect this to be a
big issue, though.
- User jdoe : this will still be available on dev builds, on both the
"read-only" and "read-write" roles that I suggested above.
I appreciate that you are all busy, so, I'll assume you are OK with the
proposed changes if I don't hear from you until next Friday :)
- Juca.
10 years
Hawkular-Archive - New Org
by Stefan Negrea
Hello,
I would like to propose a new organization in Github to host projects that
no longer receive development. "Hawkular-Archive" seems a good name for
this purpose. The goal is to keep the main Hawkular org focused on
important projects and at the same time preserve work that was already done
but away from the main organization.
>From a quick look at the current organization here are two repositories
that can be moved right away: hawkular-metrics-openshift (Openshit 2.x
cartridge for Hawkular Metrics 0.2.7 or earlier) and hawkular-bus (code
moved to another repo). Am I am sure we can find other repositories to
move.
In the long run we can develop some criteria for archiving projects but for
now we can just do a one-time major cleanup.
Any repositories that should be archived right away? Any other suggestions
for a name? Any thoughts on the idea in general?
Thank you,
Stefan Negrea
Software Engineer
10 years
Is there the feature to provide timegap between ripple restart of WildFly(JBoss EAP) cluster in Hawkular?
by Jong Seok Won
Hi all~,
Is there the feature to provide timegap between ripple restart of
WildFly(JBoss EAP) cluster in Hawkular?
If there is no feature, I'm curious there is a roadmap for this feature.
Actually, this kind of feature is useful in a production environment.
I've already checked RHQ(JON) however there is no for ripple restart.
RHQ(JON) provides below in restart operation page of Group menu.
1. "Execute: in the order specified below
2. "Halt on Failure?"
Thanks in advance.
--
Kindly Regards,
Ted Won
10 years
communty distribution question
by Jay Shaughnessy
This relates to the "[Hawkular-dev] Future Packaging of Hawkular"
thread. I started this thread to discuss a specific scenario that I
don't know how to handle.
In addition to the components listed in that e-mail, there are a few
other things in the Hawkular repo that weren't mentioned. For example,
we have pinger and we have the inventory-event bus listener. I point
out those two because I think they now exist solely to support the
community UI. URL monitoring and Alert Center do not carry over to
MIQ. With respect to the alert center we have out-of-box trigger
definitions that are currently still active and are defined specifically
to support the UI Alert center in Hawkular classic. And so we have a
situation where code I would otherwise remove may need to exist to
support the UI in the community distribution. Do we really want to
keep a community-edition UI and the server-code that *only* supports
what it does? If we remove it it will severely cripple the UI. But I
fear that the UI will anyway quickly deteriorate unless we make
concerted efforts to maintain compatible server code. Even today I
think the UI is starting to falter. I'm not sure but when I tried to
use the master version today to perform a deployment, it just hung.
So in line with Juca's questioning, what sort of effort do we put into
the community UI? A lot, and keep it working as best as we can and
ensure we mark server code that is relevant to community UI support?
None, and just make the server code be the headless provider it needs to
be to support MIQ, or something in the middle (which usually ends up
being a waste of time).
10 years
Future Packaging of Hawkular
by Stefan Negrea
Hello Everybody,
With a growing community and an increasingly large number of sub-projects,
it is the perfect time to rethink the packaging of Hawkular. This email is
a blueprint for changes to come to Hawkular as well as establish some
patterns for futures decisions.
*Motivation*
So far we have created Hawkular "All-in-one" releases that contain
everything, Hawkular server components, UI, the agent and also embedded
versions of Cassandra and KeyCloak. It's certainly the case that for
community we will still deliver a fully featured all-in-one package. We
also need to consider separate distributions for other projects that
integrate with Hawkular.
The recent effort to integrate with ManageIQ comes with its own set of
requirements for services. Not all of the Hawkular components are needed
and required components may be configured differently. For example the UI
is supplied from within ManageIQ side and Hawkular is to be run headless.
Other components like Cassandra will not run embedded, so there is no need
to supply it in an embedded package. KeyCloak will probably not be needed
because we only have one technical user that will access Hawkular services.
*Components / Services*
Here a list of projects that are part of the Hawkular organization:
- Metrics - scalable, asynchronous, multi tenant, long term metrics
storage engine
- Inventory - registry of "things" that contains info about your
applications, servers, etc. and also keeps track of their relationships
with each other
- Alerts - alerting engine that allows trigger definitions to evaluate
incoming data, generate alerts (or events), and react with flexible actions
ond lifecycle management
- CommandGW - messaging framework for communicating between components
- Bus - messaging framework for communicating between components
- BTM - provide capabilities to monitor the flow of a business
transaction instance and enable performance analysis of the individual
components that make up an application
- Agent - used to monitor WildFly and related projects
- Accounts - is the user/organization module that provides
authentication, authorization and configuration for accounts
- Data Mining - alert predictions based on time series data
- Hawkular UI
- Hawkular Charts - Angular Directives for Metrics Visualization
- Embedded Cassandra - embedded Cassandra server specifically configured
for project use
There are also various clients like the Ruby Gem or the Android client and
supporting infrastructure projects. Those are not part of this discussion.
*Options for **P**ackages*
The distribution packages need to follow product integrations with a
special distribution for community. The community package could potentially
include every single package from the Hawkular organization. However, the
rest of the packages need to be created along the lines of product use and
include only the minimal amount of components, services and third-party
libraries to satisfy integration requirements.
The distributions should be seen as funnels, the Hawkular community creates
a set of sub-projects that then get combined into few distributions that in
turn get consumed. Following this analogy, the number of packaged
distributions should also be kept to a minimum possible. For now we settled
on 3 packages detailed below, all will be available for download on
JBoss.org.
With the exception of the community distribution:
1) the QA team will be engaged in testing the package
2) the sub-components will strive to be aligned in terms of tech stack and
maturity
3) an automated test suite is required for packages that combine more than
one project
*1. Hawkular - Community Distribution*
Will include every single service listed above. The UI is a requirement
because some services are very hard to understand or use without a friendly
interface. The UI will be a community only effort at this point since there
are no plans for productization.
The current Hawkular repository will be adjusted to reflect this new
mission; the current UI will be moved to a separate repository. This
package builds on Hawkular Core Services as described next.
*2. Hawkular **Core** S**ervices*
Geared towards MiQ integration, it will include only components needed for
the MiQ provider. This package uses the Metrics Distribution as described
below.
Components included: Metrics, Alerts, Inventory, CommandGW, and Bus
Reasons for some exclusions: Keycloak (different security model), Embedded
Cassandra (only full C* deployments will be supported), and no
Hawkular UI because
it will use a special purpose UI for MiQ.
*3. **Metrics** D**istribution*
Geared towards usage as a pure Time Series Database (TSDB) and will be
exclusively made of Hawkular Metrics service.
*Expansion and Contraction*
Since its inception Hawkular has been growing by all metrics: projects,
lines of code, binary downloads, contributors, etc. But uncontrolled
increase in some metrics can be detrimental to the organization. This is
especially true for the number of projects. We will always need new
projects to experiment, but the reverse might be true for mature and
established projects. As the packaging and requirements for each
distribution mature we need to consider the reverse, merging separate
sub-projects.
*Packaging Format*
This email did not touch on the physical packaging formats (rpm, zip, war,
jar) for the 3 distributions. We will follow-up with additional details in
the coming weeks.
All feedback is more than welcomed so feel free to reach us (Heiko and
Stefan) directly or reply to this thread.
Heiko & Stefan
10 years
UI build problem on windows
by Gary Brown
Hi
Someone has reported an issue building the BTM UI on windows:
[INFO] > bufferutil(a)1.2.1 install D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\h
awkular-btm-master\hawkular-btm-master\ui\target\gulp-build\node_modules\gulp-pr
otractor\node_modules\protractor\node_modules\selenium-webdriver\node_modules\ws
\node_modules\bufferutil
[INFO] > node-gyp rebuild
[INFO]
[INFO]
[INFO] D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\hawkular-btm-master\hawkular
-btm-master\ui\target\gulp-build\node_modules\gulp-protractor\node_modules\protr
actor\node_modules\selenium-webdriver\node_modules\ws\node_modules\bufferutil>if
not defined npm_config_node_gyp (node "D:\Hawkular_SetUp_New\Latest_Hawkular\gb
rown\hawkular-btm-master\hawkular-btm-master\ui\target\gulp-build\node\node_modu
les\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )
else (node rebuild )
[ERROR] gyp ERR! configure error
[ERROR] gyp ERR! stack Error: Can't find Python executable "python", you can set
the PYTHON env variable.
[ERROR] gyp ERR! stack at failNoPython (D:\Hawkular_SetUp_New\Latest_Hawkula
r\gbrown\hawkular-btm-master\hawkular-btm-master\ui\target\gulp-build\node\node_
modules\npm\node_modules\node-gyp\lib\configure.js:116:14)
[ERROR] gyp ERR! stack at D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\hawku
lar-btm-master\hawkular-btm-master\ui\target\gulp-build\node\node_modules\npm\no
de_modules\node-gyp\lib\configure.js:71:11
[ERROR] gyp ERR! stack at FSReqWrap.oncomplete (fs.js:82:15)
[ERROR] gyp ERR! System Windows_NT 6.1.7601
[ERROR] gyp ERR! command "D:\\Hawkular_SetUp_New\\Latest_Hawkular\\gbrown\\hawku
lar-btm-master\\hawkular-btm-master\\ui\\target\\gulp-build\\node\\node.exe" "D:
\\Hawkular_SetUp_New\\Latest_Hawkular\\gbrown\\hawkular-btm-master\\hawkular-btm
-master\\ui\\target\\gulp-build\\node\\node_modules\\npm\\node_modules\\node-gyp
\\bin\\node-gyp.js" "rebuild"
[ERROR] gyp ERR! cwd D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\hawkular-btm-m
aster\hawkular-btm-master\ui\target\gulp-build\node_modules\gulp-protractor\node
_modules\protractor\node_modules\selenium-webdriver\node_modules\ws\node_modules
\bufferutil
[ERROR] gyp ERR! node -v v4.2.2
[ERROR] gyp ERR! node-gyp -v v3.0.3
[ERROR] gyp ERR! not ok
Seems like a known issue (e.g. https://github.com/niallobrien/generator-gulp-foundation/issues/13) with some possible workarounds, by installing various other components, but was just wondering if anyone knows of a better/simple workaround? Not sure if we can remove the use of node-gyp as I guess it is used by the frontend-maven-plugin?
Regards
Gary
10 years
Hawkular Inventory 0.15.0.Final Released
by Lukas Krejci
Hi all,
I'm happy to announce the release of Hawkular Inventory 0.15.0.Final.
This release was brewing for a long time and brings 1 important and big
feature - inventory synchronization for feeds.
To ensure only the minimal amount of work is done when syncing, feeds,
resource and metric types as well as resources and metrics have now an
associated "identity hash", which is a Merkle tree hash of the entity's ID,
important data (depends on entity type) and the hashes of its children. This
way one can quickly check if a feed reported any new changes since the last
time - just compare its identity hash with the last value known to you.
You can now describe the structure of the inventory a feed wants to report
locally on the feed and report it back to inventory server - it will ensure
that all creates, updates and deletes are applied so that inventory reflects
what the feed "sees".
Apart from this there's been a handful of bugfixes and improvements and
updates to REST documentation (that don't seem to be reflected on hawkular.org
yet).
In the future releases, we plan to refine the inventory sync further,
restructure the REST API to get rid of the potential ambiguities and many
other things.
Special thanks go out to Justine Tunney, Heiko Rupp, Jirka Kremser, Pavol
Loffay and Peter Palaga for their contributions to the release.
With kind regards,
--
Lukas Krejci
10 years
