[keycloak-dev] Configure surefire to run test classes in static order?

[Marek Posolda]

Sent PR for this: https://github.com/keycloak/keycloak/pull/6244

There is some chance, that some tests, which were passing until now will 
start to fail (or viceversa). But from the long-term perspective, things 
should be better and more predictable hopefully :)

Will see if travis and jenkins will pass...

Marek

On 15. 08. 19 15:19, Marek Posolda wrote:
> Thanks everyone, I've created JIRA and will try to take a look at it: 
> https://issues.jboss.org/browse/KEYCLOAK-11122
>
> Marek
>
> On 15. 08. 19 12:48, Bruno Oliveira wrote:
>> +1 on this. That may also help us also with CrossDC tests.
>>
>> On Wed, Aug 14, 2019 at 5:33 PM Marek Posolda <mposolda at redhat.com> 
>> wrote:
>>> It seems it may be possible to configure maven-surefire-plugin to use
>>> "static" order of test classes when executing testsuite. Should be
>>> doable with the "runOrder" parameter switched for example to
>>> "alphabetical" [1].
>>>
>>> In theory, order of the tests shouldn't be important as each test 
>>> should
>>> cleanup after itself. Reality is, that we often have bugs due that some
>>> test class didn't properly clean everything after itself. And when some
>>> following test fails, it may not be so trivial to find the original 
>>> test
>>> class, which caused this. Static order should help with having the
>>> testsuite more deterministic and ensure that you can simulate same
>>> issues locally and in jenkins - see for example this comment for some
>>> details [2].
>>>
>>> WDYT?
>>>
>>> [1]
>>> http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder 
>>>
>>> [2]
>>> https://issues.jboss.org/browse/KEYCLOAK-10904?focusedCommentId=13770826&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13770826 
>>>
>>>
>>> Marek
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>>
>