10 a.m.
On Tue, Feb 1, 2022 at 8:43 AM Jan Kašík <jkasik(a)redhat.com> wrote:
I took a look at the PoC and tried to run it with Podman (IIRC officially unsupported by
TestContainers so mileage may vary). It took up to 30 seconds to initialize containers
already downloaded from quay.io for a test class. I'm also not a big fan of a common
parent class for each test class and I would prefer to have an additional few lines of
code in each test class. Otherwise it runs fine.
Once the containers have been downloaded from quay.io, the startup times
are pretty fast on my machine. The browser and HAL container are started just
once for all tests, which takes about 20 seconds. The WildFly instance which
is started once for each test takes about another 20 seconds.
Since the test should run on a CI environment, the startup times don't really
matter IMO.
I agree that we should avoid putting too much logic in common test classes.
But I think there are useful for several reasons:
- place common annotations
- start a clean WildFly instance
Thanks to having multiple Maven modules with tests it is currently
possible to parallelize the run of individual modules in Jenkins matrix configuration. Is
it possible to download all needed containers in a preparation step to not have to do this
in every matrix configuration? Initialization like this would add some execution time to
the test run.
I'm not sure if that's possible.
IMHO it is hard to say if stability will improve in the usual large
scale HAL tests are running in. Jenkins nodes we use are virtualized. I can imagine that
using TestContainers may improve stability in terms of dependencies - now we would be sure
browser and VNC are always present and properly configured but that's not a common
problem from my experience. How would using TestContainers improve stability?
When I run the tests locally, the tests are not very stable and I often run
into timeouts. Also the tests interfere with other applications. This might
not be a problem in a CI environment though.
If all the changes you mentioned are aimed only at upstream (since
you mention GH actions and TeamCity), would it be possible to still run tests without
TestContainers?
Does the TestContainers solution suppose to replace existing testsuite?
Currently the tests depend on Testcontainers and would not run without it.
However I'd like the tests to be independent of Testcontainers. They should just
depend on the pages, fragments, helper classes, Arquillian and JUnit.
I don't think it makes sense to rewrite the current test suite. We put too
much effort into it. If you think the new approach makes sense, we should
rather create a branch and apply the new approach into the existing test suite.
WDYT?
--
Harald Pehl
Senior Software Engineer
hpehl(a)redhat.com