[infinispan-dev] Uber jars testing

[Galder Zamarreno]

Good post Jiri, this got me thinking :)

Running the entire testsuite again with uber jars would add a lot of time to the build time.

Maybe we should have a set of tests that must be executed for sure, e.g. like Wildfly's smoke tests [1]. We have "functional" group but right now it covers pretty much all tests.

Such tests should live in a separate testsuite, so that we could add the essential tests for *all* components. In a way, we've already done some of this in integrationtests/ but it's not really well structured for this aim.

Also, if we would go down this path, something we should take advantage of (if possible with JUnit/TestNG) is what Gustavo did with the Spark tests in [2], where he used suites to make it faster to run things, by starting a cache manager for distributed caches, running all distributed tests...etc. In a way, I think we can already do this with Arquillian Infinispan integration, so Arquillian would probably well suited for such smoke testsuite.

Thoughts?

Cheers,

[1] https://github.com/wildfly/wildfly#running-the-testsuite
[2] https://github.com/infinispan/infinispan-spark/tree/master/src/test/scala/org/infinispan/spark
--
Galder Zamarreño
Infinispan, Red Hat

----- Original Message -----
> Hi Jiri, comments inline.
> 
> On 2.9.2015 10:40, Jiri Holusa wrote:
> > Hi all,
> >
> > we've been thinking for a while, how to test ISPN uber jars. The current
> > status is that we actually don't have many tests in the testsuite, there
> > are few tests in integrationtests/all-embedded-* modules that are
> > basically copies of the actual tests in corresponding modules. We think
> > that this test coverage is not enough and more importantly, they are
> > duplicates.
> >
> > The questions are now following:
> > * which tests should be invoked with uber-jars? Whole ISPN testsuite? Only
> > integrationtests module?
> 
> The goal is to run the whole test suite because, as you said, we don't
> have enough tests in integrationtests/*   And we can't duplicate all
> test classes from individual modules here.
> 
> > * how would it run? Create Maven different profiles for "classic" jars and
> > uber jars? Or try to use some Maven exclusion magic if even possible?
> >
> > Some time ago, we had discussion about this with Sebastian, who suggested
> > that running only integrationtests module would be sufficient, because
> > uber-jars are really about packaging, not the functionality itself. But I
> > don't know if the tests coverage is sufficient in that level, I would be
> > much more confident if we could run the whole ISPN testsuite against
> > uber-jars.
> Right. Uber-jars are about packaging but you don't know that the
> packiging is right until you try all the features and see that
> everything works. There might be some classes missing (just for some
> particular features), same classes in different packages, the
> Manifest.mf might be corrupted and then something won't work in OSGi.
> 
> 
> I'd prefer a separate Maven profile. IMO, exclusions are too error-prone.
> 
> 
> Martin
> >
> > I'm opening this for wider discussion as we should agree on the way how to
> > do it, so we could do it right :)
> >
> > Cheers,
> > Jiri
> >
> >
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>