[keycloak-dev] Configure surefire to run test classes in static order?

[Marek Posolda]

It seems it may be possible to configure maven-surefire-plugin to use 
"static" order of test classes when executing testsuite. Should be 
doable with the "runOrder" parameter switched for example to 
"alphabetical" [1].

In theory, order of the tests shouldn't be important as each test should 
cleanup after itself. Reality is, that we often have bugs due that some 
test class didn't properly clean everything after itself. And when some 
following test fails, it may not be so trivial to find the original test 
class, which caused this. Static order should help with having the 
testsuite more deterministic and ensure that you can simulate same 
issues locally and in jenkins - see for example this comment for some 
details [2].

WDYT?

[1] 
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder
[2] 
https://issues.jboss.org/browse/KEYCLOAK-10904?focusedCommentId=13770826&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13770826

Marek