[keycloak-dev] Configure surefire to run test classes in static order?

[Pavel Drozd]

+1

On Thu, Aug 15, 2019 at 9:12 AM Peter Skopek <pskopek at redhat.com> wrote:

> I agree to set runOrder to "alphabetical". This will unify test runs
> on local machines and Jenkins which is clear benefit for us.
> Peter
>
> On Wed, Aug 14, 2019 at 10:35 PM Marek Posolda <mposolda at redhat.com>
> wrote:
> >
> > It seems it may be possible to configure maven-surefire-plugin to use
> > "static" order of test classes when executing testsuite. Should be
> > doable with the "runOrder" parameter switched for example to
> > "alphabetical" [1].
> >
> > In theory, order of the tests shouldn't be important as each test should
> > cleanup after itself. Reality is, that we often have bugs due that some
> > test class didn't properly clean everything after itself. And when some
> > following test fails, it may not be so trivial to find the original test
> > class, which caused this. Static order should help with having the
> > testsuite more deterministic and ensure that you can simulate same
> > issues locally and in jenkins - see for example this comment for some
> > details [2].
> >
> > WDYT?
> >
> > [1]
> >
> http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder
> > [2]
> >
> https://issues.jboss.org/browse/KEYCLOAK-10904?focusedCommentId=13770826&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13770826
> >
> > Marek
> >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>