11:09 p.m.
I am currently working on accounts integration in metrics. The integration code lives in
the hawkular-component module in the metrics repo. Now I want to write some integration
tests to verify that things work as expected. I don’t really think that integration/glue
code belongs in the component repo. I want to make sure that wherever it lives I can still
get feed back from CI builds when changes are made. When I make a change to component code
in the metrics repo, the CI build runs and checks for regressions by running tests. There
is a pretty quick feedback loop which is very important. If the glue code is living
elsewhere, I would like for a a CI build to get kicked off at some point after the metrics
CI builds completes successfully so that I can find out if any of my changes to the
component code caused a regression in the glue code. And of course if I make a change
directly in the glue code, I should get the feedback from the check-in CI job. In RHQ we
had a build pipeline with jenkins that handled this for us. Is this possible with travis?
For now I am putting the glue code and tests in the metrics repo so that I can have the CI
feedback loop, but I see this as a short term solution. It may work fine in this instance,
but what about when I want to test some other integration? I will be pulling in more
hawkular dependencies and eventually hit circular dependencies. And the quick feedback
that I get from the component CI build will get a lot slower in the process.
- John
2:49 a.m.
----- Original Message -----
From: "John Sanda" <jsanda(a)redhat.com>
To: "Discussions around Hawkular development"
<hawkular-dev(a)lists.jboss.org>
Sent: Wednesday, January 13, 2016 6:09:20 AM
Subject: [Hawkular-dev] where should integration/glue code and tests live?
I am currently working on accounts integration in metrics. The integration
code lives in the hawkular-component module in the metrics repo. Now I want
to write some integration tests to verify that things work as expected. I
don’t really think that integration/glue code belongs in the component repo.
I want to make sure that wherever it lives I can still get feed back from CI
builds when changes are made. When I make a change to component code in the
metrics repo, the CI build runs and checks for regressions by running tests.
There is a pretty quick feedback loop which is very important. If the glue
code is living elsewhere, I would like for a a CI build to get kicked off at
some point after the metrics CI builds completes successfully so that I can
find out if any of my changes to the component code caused a regression in
the glue code. And of course if I make a change directly in the glue code, I
should get the feedback from the check-in CI job. In RHQ we had a build
pipeline with jenkins that handled this for us. Is this possible with
travis?
For now I am putting the glue code and tests in the metrics repo so that I
can have the CI feedback loop, but I see this as a short term solution. It
may work fine in this instance, but what about when I want to test some
other integration? I will be pulling in more hawkular dependencies and
eventually hit circular dependencies. And the quick feedback that I get from
the component CI build will get a lot slower in the process.
I think this topic have been discussed and might have several angles as integration test
can have a wide meaning.
Today there is a distribution wildfly+bus+accounts that can be used for small itests, for
example in alerts we use that to check rest and bus connectivity as these tests are focus
mainly on the component.
Other broader integration tests should live on hawkular (I think).
9:47 a.m.
On 13 Jan 2016, at 6:09, John Sanda wrote:
I am currently working on accounts integration in metrics. The
integration code lives in the hawkular-component module in the metrics
repo. Now I want to write some integration tests to verify that things
work as expected. I don’t really think that integration/glue code
belongs in the component repo. I want to make sure that wherever it
lives I can still get feed back from CI builds when changes are made.
When I make a change to component code in the metrics repo, the CI
build runs and checks for regressions by running tests. There is a
pretty quick feedback loop which is very important. If the glue code
is living elsewhere, I would like for a a CI build to get kicked off
at some
is that metrics-accounts specific glue code? Which I mean
as is that needed also for hawkular? If so, is that test
then enough to verify functioning for all of Hawkular?
point after the metrics CI builds completes successfully so that I
can
find out if any of my changes to the component code caused a
regression in the glue code. And of course if I make a change directly
in the glue code, I should get the feedback from the check-in CI job.
In RHQ we had a build pipeline with jenkins that handled this for us.
Is this possible with travis?
Travis apparently does not have them yet. They seem
in the pipeline, but it looks like in the first iteration they
will not be as powerful as in Jenkins.
In theory we could that with Travis by having an overall
repo/build script
foo:
- build metrics
- test integration
where 'build metrics' would push the generated artifact somewhere
and 'test integration' would load if from this somewhere and run the
tests against it. IIrc we already push all the build results
to the Nexus snaphshot repo, so we already have the 'somewhere'.
For some "larger scale" itests, I think they should go into
hawkular-main, but I understand very well that this does not
solve your issue especially as there is currently no way of
triggering a hawkular-main build when e.g. metric makes a
change.
We need to keep an eye though on the duplication of code
to standup the itests. It may not be good to have that in all
projects in various permutations, as it will be a maintenance
nightmare.
10:17 a.m.
On Jan 13, 2016, at 10:47 AM, Heiko W.Rupp <hrupp(a)redhat.com>
wrote:
On 13 Jan 2016, at 6:09, John Sanda wrote:
> I am currently working on accounts integration in metrics. The
> integration code lives in the hawkular-component module in the metrics
> repo. Now I want to write some integration tests to verify that things
> work as expected. I don’t really think that integration/glue code
> belongs in the component repo. I want to make sure that wherever it
> lives I can still get feed back from CI builds when changes are made.
> When I make a change to component code in the metrics repo, the CI
> build runs and checks for regressions by running tests. There is a
> pretty quick feedback loop which is very important. If the glue code
> is living elsewhere, I would like for a a CI build to get kicked off
> at some
is that metrics-accounts specific glue code? Which I mean
as is that needed also for hawkular? If so, is that test
then enough to verify functioning for all of Hawkular?
The glue code is for adding authentication support via accounts. This code is not included
in either of the stand alone or OpenShift metric’s distros. It is only included in the
metrics WAR that is included in the full hawkular server.
I do not think the test would be enough to verify that the whole server is functioning,
although I think it would be easy enough to write similar tests for other hawkular
components that is of course once the test is actually written :)
> point after the metrics CI builds completes successfully so that I can
> find out if any of my changes to the component code caused a
> regression in the glue code. And of course if I make a change directly
> in the glue code, I should get the feedback from the check-in CI job.
> In RHQ we had a build pipeline with jenkins that handled this for us.
> Is this possible with travis?
Travis apparently does not have them yet. They seem
in the pipeline, but it looks like in the first iteration they
will not be as powerful as in Jenkins.
In theory we could that with Travis by having an overall
repo/build script
foo:
- build metrics
- test integration
where 'build metrics' would push the generated artifact somewhere
and 'test integration' would load if from this somewhere and run the
tests against it. IIrc we already push all the build results
to the Nexus snaphshot repo, so we already have the 'somewhere'.
For some "larger scale" itests, I think they should go into
hawkular-main, but I understand very well that this does not
solve your issue especially as there is currently no way of
triggering a hawkular-main build when e.g. metric makes a
change.
We need to keep an eye though on the duplication of code
to standup the itests. It may not be good to have that in all
projects in various permutations, as it will be a maintenance
nightmare.
Totally agree which is another reason I tend to think that test which exercise hawkular
component integration probably do not belong in the component repos.
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
5:05 a.m.
I don't think a build pipeline would help in this case, as the Hawkular
pom references metrics with either a released or a srcdep version.
If you need to check your current Metrics work doesn't cause any
regression in Hawkular, you can create a branch in Metrics (instead of a
simple PR), create a PR in Hawkular and update the srcdep version with
your branch latest commit.
Then Travis will build and run Hawkular tests.
Le 13/01/2016 17:17, John Sanda a écrit :
> On Jan 13, 2016, at 10:47 AM, Heiko W.Rupp <hrupp(a)redhat.com> wrote:
>
> On 13 Jan 2016, at 6:09, John Sanda wrote:
>
>> I am currently working on accounts integration in metrics. The
>> integration code lives in the hawkular-component module in the metrics
>> repo. Now I want to write some integration tests to verify that things
>> work as expected. I don’t really think that integration/glue code
>> belongs in the component repo. I want to make sure that wherever it
>> lives I can still get feed back from CI builds when changes are made.
>> When I make a change to component code in the metrics repo, the CI
>> build runs and checks for regressions by running tests. There is a
>> pretty quick feedback loop which is very important. If the glue code
>> is living elsewhere, I would like for a a CI build to get kicked off
>> at some
>
> is that metrics-accounts specific glue code? Which I mean
> as is that needed also for hawkular? If so, is that test
> then enough to verify functioning for all of Hawkular?
The glue code is for adding authentication support via accounts. This code is not
included in either of the stand alone or OpenShift metric’s distros. It is only included
in the metrics WAR that is included in the full hawkular server.
I do not think the test would be enough to verify that the whole server is functioning,
although I think it would be easy enough to write similar tests for other hawkular
components that is of course once the test is actually written :)
>
>> point after the metrics CI builds completes successfully so that I can
>> find out if any of my changes to the component code caused a
>> regression in the glue code. And of course if I make a change directly
>> in the glue code, I should get the feedback from the check-in CI job.
>> In RHQ we had a build pipeline with jenkins that handled this for us.
>> Is this possible with travis?
>
> Travis apparently does not have them yet. They seem
> in the pipeline, but it looks like in the first iteration they
> will not be as powerful as in Jenkins.
> In theory we could that with Travis by having an overall
> repo/build script
> foo:
> - build metrics
> - test integration
>
> where 'build metrics' would push the generated artifact somewhere
> and 'test integration' would load if from this somewhere and run the
> tests against it. IIrc we already push all the build results
> to the Nexus snaphshot repo, so we already have the 'somewhere'.
>
> For some "larger scale" itests, I think they should go into
> hawkular-main, but I understand very well that this does not
> solve your issue especially as there is currently no way of
> triggering a hawkular-main build when e.g. metric makes a
> change.
>
> We need to keep an eye though on the duplication of code
> to standup the itests. It may not be good to have that in all
> projects in various permutations, as it will be a maintenance
> nightmare.
Totally agree which is another reason I tend to think that test which exercise hawkular
component integration probably do not belong in the component repos.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
5:26 a.m.
In my opinion, there has to be tests in the components testing the
"public API", ie, integration points that other components might use. In
Accounts' case, it needs to have tests that assert that, for instance,
the permission checker works as per the contract. The test itself is my
"consumer", playing the role of a downstream component.
In Hawkular (aka: the ultimate glue), there has to be tests that assert
workflows, probably exercising multiple components. Those would be used
to accept or reject a PR.
So, my answer is: there has to be itests in all components, plus "real
workflow" tests on Hawkular, possibly even doing things on the UI.
- Juca.
5:37 a.m.
+1 on all points
Le 15/01/2016 12:26, Juraci Paixão Kröhling a écrit :
In my opinion, there has to be tests in the components testing the
"public API", ie, integration points that other components might use. In
Accounts' case, it needs to have tests that assert that, for instance,
the permission checker works as per the contract. The test itself is my
"consumer", playing the role of a downstream component.
In Hawkular (aka: the ultimate glue), there has to be tests that assert
workflows, probably exercising multiple components. Those would be used
to accept or reject a PR.
So, my answer is: there has to be itests in all components, plus "real
workflow" tests on Hawkular, possibly even doing things on the UI.
- Juca.
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
3235
days inactive
3237
days old
6 comments
5 participants
participants (5)
-
Heiko W.Rupp -
John Sanda -
Juraci Paixão Kröhling -
Lucas Ponce -
Thomas Segismont