[infinispan-dev] HotRod client TCK

[Martin Gencur]

Hi,
thanks for looking at the list of tests and thanks for suggestions. 
We'll incorporate them in the final list of tests.

What Radim suggest has some advantages and some drawbacks, but I see 
this as an addition to the client TCK.
This approach can verify that the client sends some predefined commands 
with predefined values but does that really verify that the user will 
get the expected results? I'm not so sure. There can be some client-side 
logic that does other modifications. Here I see room for a lot of missed 
bugs.

I'd say we need real client-side tests which verify the client behavior 
from the user perspective. Let me also add that I see the Java client 
test suite as an etalon (reference standard). The server side behavior 
has been tested through the Java test suite and other clients don't need 
to test that again, IMO. The goal is to test the client-side. Having a 
pre-defined configuration for server that would be used in all client 
implementation tests should provide some common ground for the tests.

As to the real TCK suggested by Gustavo, I remember the discussion and 
we discussed that at the clustering meeting last year.
Since the Java HotRod client test suite has about 1500 tests (maybe more 
now?) we would need to rewrite all the tests in the new language. And 
I'm not sure running those tests with various implementations would be 
without problems. I'd love to see this working but I'm afraid that we 
don't have time and resources to do this any time soon.

Martin

On 8.5.2017 13:32, Galder Zamarreño wrote:
> I think in general it'd be a good idea to try to verify somehow most of the TCK via some server-side logic, as Radim hinted, and where that's not possible, revert to just verifying the client has tests to cover certain scenarios.