[infinispan-dev] Adding tests for new cache mode

[Radim Vansa]

Uuuh... if it was just so easy. Now I've played with @Factory, 
@DataProvider and others (since we want to parameterize whole class, not 
just invocation of method), and it's broken - I can't simply rename the 
test according to parameters, it behaves differently in IntelliJ and on 
command line and also with different versions of TestNG. But that's just 
aesthetic concern.

Another problem is the resource tracking AbstractInfinispanTest uses:

@BeforeTest/AfterTest is ran only once per class, not per instance
@BeforeClass/AfterClass is ran once per instance, but in a loop on the 
instances within the same thread, before the methods are executed

The tests are then run in the order instance1.test1(), 
instance2.test1(), instance1.test2() - so this needs to be reordered 
using IMethodInterceptor. Luckily, the BeforeClass invocation is lazy 
and AfterClass is eager, so these methods are invoked when these should be.

So, at this point [1] I managed to get the tests running fine, and if I 
omit the testName in @Test, it is reported correctly in 
target/surefire-reports/. However, the output that's printed when I run 
maven test does not involve the parameter, and in IntelliJ the test 
subsequent result gets just marked with (x) where x is some number. When 
testName is set, it's repeated for all results in reports, in IntelliJ 
as well (with (x) for each method invocation), and commandline output is 
not changed.

If anyone knows how to fix that, go for it - I don't know what IntelliJ 
or the surefire reporter picks. Just a hint - ITest interface won't help 
you, that just spoils everything.

Radim

[1] https://github.com/rvansa/infinispan/tree/t_test_factory

On 05/18/2016 05:02 PM, Dan Berindei wrote:
> On Tue, May 17, 2016 at 3:26 PM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>> On 17/05/2016 14:10, Radim Vansa wrote:
>>> Hi,
>>>
>>> I've decided to start working on Scattered Cache [1][2] POC. I'd like to
>>> use most of the tests for distributed mode, but just extending
>>> DistXxxTest with ScatteredXxxTest and overriding getCacheMode() seems
>>> quite inelegant, though this is a common practice for repl/dist tests. I
>>> had similar problem with Simple Cache, but I didn't need as many tests
>>> for that.
>>>
>>> @Parameters are not used as much in our testsuite - is there any reason
>> Not using @Parameters is a mistake, IMHO, so if you're willing to
>> convert the relevant ones, that would be lovely.
>>
> +1 to switch to @Parameters as many tests as you want!
>
> Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team