On Tue, Feb 1, 2022 at 8:43 AM Jan Kašík <jkasik@redhat.com> wrote:
>
> I took a look at the PoC and tried to run it with Podman (IIRC officially unsupported by TestContainers so mileage may vary). It took up to 30 seconds to initialize containers already downloaded from quay.io for a test class. I'm also not a big fan of a common parent class for each test class and I would prefer to have an additional few lines of code in each test class. Otherwise it runs fine.
Once the containers have been downloaded from quay.io, the startup times
are pretty fast on my machine. The browser and HAL container are started just
once for all tests, which takes about 20 seconds. The WildFly instance which
is started once for each test takes about another 20 seconds.
I observed initialization taking place with each test class.
Since the test should run on a CI environment, the startup times don't really
matter IMO.
IMO it can add up to tens of minutes to the run.
I agree that we should avoid putting too much logic in common test classes.
But I think there are useful for several reasons:
- place common annotations
- start a clean WildFly instance
It always starts with a few lines of code and ends up in a pile of mess. :) I'm really not a fan of inheritance in case of test classes since I spent some time trying to resolve issues caused by this.
> Thanks to having multiple Maven modules with tests it is currently possible to parallelize the run of individual modules in Jenkins matrix configuration. Is it possible to download all needed containers in a preparation step to not have to do this in every matrix configuration? Initialization like this would add some execution time to the test run.
I'm not sure if that's possible.
> IMHO it is hard to say if stability will improve in the usual large scale HAL tests are running in. Jenkins nodes we use are virtualized. I can imagine that using TestContainers may improve stability in terms of dependencies - now we would be sure browser and VNC are always present and properly configured but that's not a common problem from my experience. How would using TestContainers improve stability?
When I run the tests locally, the tests are not very stable and I often run
into timeouts. Also the tests interfere with other applications. This might
not be a problem in a CI environment though.
> If all the changes you mentioned are aimed only at upstream (since you mention GH actions and TeamCity), would it be possible to still run tests without TestContainers?
> Does the TestContainers solution suppose to replace existing testsuite?
Currently the tests depend on Testcontainers and would not run without it.
However I'd like the tests to be independent of Testcontainers. They should just
depend on the pages, fragments, helper classes, Arquillian and JUnit.
I don't think it makes sense to rewrite the current test suite. We put too
much effort into it. If you think the new approach makes sense, we should
rather create a branch and apply the new approach into the existing test suite.
WDYT?
--
Harald Pehl
Senior Software Engineer
hpehl@redhat.com