+1
On Thu, Aug 15, 2019 at 9:12 AM Peter Skopek <pskopek(a)redhat.com> wrote:
I agree to set runOrder to "alphabetical". This will unify
test runs
on local machines and Jenkins which is clear benefit for us.
Peter
On Wed, Aug 14, 2019 at 10:35 PM Marek Posolda <mposolda(a)redhat.com>
wrote:
>
> It seems it may be possible to configure maven-surefire-plugin to use
> "static" order of test classes when executing testsuite. Should be
> doable with the "runOrder" parameter switched for example to
> "alphabetical" [1].
>
> In theory, order of the tests shouldn't be important as each test should
> cleanup after itself. Reality is, that we often have bugs due that some
> test class didn't properly clean everything after itself. And when some
> following test fails, it may not be so trivial to find the original test
> class, which caused this. Static order should help with having the
> testsuite more deterministic and ensure that you can simulate same
> issues locally and in jenkins - see for example this comment for some
> details [2].
>
> WDYT?
>
> [1]
>
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#run...
> [2]
>
https://issues.jboss.org/browse/KEYCLOAK-10904?focusedCommentId=13770826&...
>
> Marek
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev