[aerogear-dev] Integration test execution policy on Aerogear

[Karel Piwko]

Hi All,

this week we added integration tests to Unified Push Server[1]. There are 3 new
profiles: as711-managed, as711-remote, eap610-managed. Managed profiles are
intended to run in CI, everything is managed by tests. Remote profiles are for
test development - you start AS on your own and save ~10 seconds in each test
suite execution.

Now we need to reach the agreement what are *the right* defaults. Here are the
options:

1/ Now: Run integration tests in Maven verify phase by default. as711-managed is
activated by default

This means that 'mvn package' is not affected. 'mvn install' will run the
tests, starting its own AS7, that is downloaded if not available locally. So far
tests take like 2 minutes, 10 minutes Maven trying download Internet minus
porn for the first run. The problem is that you have to ensure that AS7.1 ports
(8080, 9999) are free every time you run mvn install.

2/ Keep 1/ but add an offset to ports (e.g +100). 

This will reduce the port conflicts if you are running AS instance somewhere
else. Or if you are listening to mp3 streams on 9999 ;-). This will complicate
remote scenario, e.g. you'd need to start remote instance with offset. Does not
affect CI.

3/ Skip integration tests by default. No "container" profile is activated by
default, skipITs=true by default. Activate container profile in Travis and QE
machines only. 

Would need to activate a Maven profile for test development. Travis will
still bug developers if something fails ;-)

4/ Something else?

WDYT? I'd like to reach an agreement so that the same policy could be applied
across all Aerogear modules.

Thanks,

Karel

[1]
https://github.com/aerogear/aerogear-unified-push-server/commit/e1651edb68af91a0360e0567c35bbfc08a78c73a