There is some chance, that some tests, which were passing until now will
start to fail (or viceversa). But from the long-term perspective, things
should be better and more predictable hopefully :)
Will see if travis and jenkins will pass...
Marek
On 15. 08. 19 15:19, Marek Posolda wrote:
Thanks everyone, I've created JIRA and will try to take a look at
it:
https://issues.jboss.org/browse/KEYCLOAK-11122
Marek
On 15. 08. 19 12:48, Bruno Oliveira wrote:
> +1 on this. That may also help us also with CrossDC tests.
>
> On Wed, Aug 14, 2019 at 5:33 PM Marek Posolda <mposolda(a)redhat.com>
> wrote:
>> It seems it may be possible to configure maven-surefire-plugin to use
>> "static" order of test classes when executing testsuite. Should be
>> doable with the "runOrder" parameter switched for example to
>> "alphabetical" [1].
>>
>> In theory, order of the tests shouldn't be important as each test
>> should
>> cleanup after itself. Reality is, that we often have bugs due that some
>> test class didn't properly clean everything after itself. And when some
>> following test fails, it may not be so trivial to find the original
>> test
>> class, which caused this. Static order should help with having the
>> testsuite more deterministic and ensure that you can simulate same
>> issues locally and in jenkins - see for example this comment for some
>> details [2].
>>
>> WDYT?
>>
>> [1]
>>
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#run...
>>
>> [2]
>>
https://issues.jboss.org/browse/KEYCLOAK-10904?focusedCommentId=13770826&...
>>
>>
>> Marek
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>