[aerogear-dev] PushEE testing POC

[Karel Piwko]

Seems to me that we might agree on moving tests into separate repository for
now while making them a part of src/test of the project itself once we
stabilize testing tools.

Comments inline.

Karel

On Tue, 21 May 2013 11:11:42 +0200
Matthias Wessendorf <matzew at apache.org> wrote:

> On Tue, May 21, 2013 at 9:15 AM, Karel Piwko <kpiwko at redhat.com> wrote:
> 
> > Well, we have 3 options where to put tests:
> >
> > 1/ Have them tightly integrated - that would be src/test
> > 2/ Have them next to actual code - that's current proposal
> > 3/ Have them in separate repository - that's Matthias option
> >
> > I chose 2/ because that way people are not required to setup anything to
> >
> 
> well, it's not really tied to the maven default layout;
> folks have to explictily cd into the "tests" folder, at root level;
> Also, I guess you need separated CI hooks for that
> 
Yes, we need to add one more Maven execution set up in CI.
> 
> 
> 
> > code on the component while being aware something like integration tests
> > exists and it's not that difficult to fix them if broken by your last
> > commit.
> >
> 
> fair point, but they can be always executed against the latest snapshot.

Yes, they will be no matter where the tests are put into. But "cd tests & fix"
is much less effort compared to check CI job where tests actually are, clone,
fix and rerun.
> 
> 
> 
> > That's the main reason why I do not like 3/, separate lifecycle for tests
> > adds
> > entry barrier for test development and maintenance.
> 
> 
> cd tests (at root level), make this already very similar to a separate
> project, hence it has
> an individual "lifecycle", right ?

It is a separate project. But lifecycle is shared.
> 
> 
> 
> > Option 1/ is the best if
> > the set of test tools is stable, which is unfortunately not the case yet,
> > unfortunately.
> >
> 
> For me, personally src/test is more for unit/mock testing. But that's just
> personal preference.

Right.
> 
> 
> 
> 
> >
> > Personally, I do not like integration tests being kept separately. With
> > Arquillian, the boundary between integration and unit test is very blur, so
> > so the only point keeping tests separated is developer turnaround - test
> > execution feedback should be quick. So, if you guys figure out that
> > integration
> > tests take too much time in future, I'd opt for making smoke profile with a
> > subset of selected tests and full profile for CI purposes.
> >
> > Karel
> >
> > On Mon, 20 May 2013 16:30:24 -0300
> > Bruno Oliveira <bruno at abstractj.org> wrote:
> >
> > > Hi Karel, why do not follow conventions? src/test?
> > >
> > >
> > http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
> > >
> > > Karel Piwko wrote:
> > > > Hi All,
> > > >
> > > > I've just sent a PR for PushEE testing [1]. The idea is to write tests
> > > > covering specification and simply execute those against a real running
> > > > server instance. More details at [2].
> > > >
> > > > I have evaluated multiple API approaches, described here[3], Groovy and
> > > > Spock seems to be the best to me.
> > > >
> > > > Any comments/suggestions/objections very welcomed. My plan is to start
> > > > covering specs we have so far and run it on a CI server.
> > > >
> > > > Thanks,
> > > >
> > > > Karel
> > > >
> > > > [1] https://github.com/matzew/pushee/pull/6/
> > > > [2] https://github.com/kpiwko/pushee/blob/tests/tests/readme.txt
> > > > [3] https://gist.github.com/kpiwko/5612949
> > > > _______________________________________________
> > > > aerogear-dev mailing list
> > > > aerogear-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/aerogear-dev
> > > _______________________________________________
> > > aerogear-dev mailing list
> > > aerogear-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/aerogear-dev
> >
> > _______________________________________________
> > aerogear-dev mailing list
> > aerogear-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/aerogear-dev
> >
> 
> 
>