[infinispan-dev] HotRod client TCK

Tue May 9 13:13:48 EDT 2017

On 05/08/2017 07:10 AM, Galder Zamarreño wrote:
> I think there's some value in Radim's suggestion. The email was not fully clear to me initially but after reading a few times I understood what he was referring to. @Radim, correct me if I'm wrong...
>
> Right now clients verify that they behave as expected, e.g. JS client uses its asserts, Java client uses other asserts. What Radim is trying to say is that there needs to be a way to verify they work adequately independent of their implementations.
>
> So, the only way to do that is to verify it at the server level. Not sure what exactly he means by the fake server, but more than a fake server, I'd be more inclined to modify the server to that it can somehow act as TCK verifier. This is to avoid having to reimplement transport logic, protocol decoder...etc in a new fake server.

I think you got the idea. I am not trying to push any particular 
implementation of the "fake server" - you could just tweak existing one, 
but the purest and most deterministic approach would be having a script 
that could look like:

expect connection A to serverX/any server
expect receive from A <base64 encoded bytes/hex encoded bytes>
send to A <... bytes>
expect connection A closed

Implementing a server that interprets such script isn't that complex; 
you don't have to deal with protocol decoder (what's transport logic on 
server?), because you just expect and send bytes.

Radim

>
> Cheers,
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>> On 11 Apr 2017, at 15:57, Radim Vansa <rvansa at redhat.com> wrote:
>>
>> Since these tests use real server(s), many of them test not only the
>> client behaviour (generating correct commands according to the
>> protocol), but server, too. While this is practical (we need to test
>> server somehow, too), there's nothing all the tests across languages
>> will have physically in common and all comparison is prone to human error.
>>
>> If we want to test various implementations of the client, maybe it would
>> make sense to give the clients a fake server that will have just a
>> scenario of expected commands to receive and pre-defined responses. We
>> could use audit log to generate such scenario based on the actual Java
>> tests.
>>
>> But then we'd have to test the actual behaviour on server, and we'd need
>> a way to issue the commands.
>>
>> Just my 2c
>>
>> Radim
>>
>> On 04/11/2017 02:33 PM, Martin Gencur wrote:
>>> Hello all,
>>> we have been working on https://issues.jboss.org/browse/ISPN-7120.
>>>
>>> Anna has finished the first step from the JIRA - collecting information
>>> about tests in the Java HotRod client test suite (including server
>>> integration tests) and it is now prepared for wider review.
>>>
>>> She created a spreadsheet [1]. The spread sheet includes for each Java
>>> test its name, the suggested target package in the TCK, whether to
>>> include it in the TCK or not, and some other notes. The suggested
>>> package also poses grouping for the tests (e.g. tck.query, tck.near,
>>> tck.xsite, ...)
>>>
>>> Let me add that right now the goal is not to create a true TCK [2]. The
>>> goal is to make sure that all implementations of the HotRod protocol
>>> have sufficient test coverage and possibly the same server side of the
>>> client-server test (including the server version and configuration).
>>>
>>> What are the next step?
>>>
>>> * Please review the list (at least a quick look) and see if some of the
>>> tests which are NOT suggested for the TCK should be added or vice versa.
>>> * I suppose the next step would then be to check other implementations
>>> (C#, C++, NodeJS, ..) and identify tests which are missing there (there
>>> will surely be some).
>>> * Gradually implement the missing tests in the other implementations
>>>     Note: Here we should ensure that the server is configured in the same
>>> way for all implementations. One way to achieve this (thanks Anna for
>>> suggestion!) is to have a shell/batch scripts for CLI which would be
>>> executed before the tests. This can probably be done for all impls. and
>>> both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes
>>> useless because it uses Creaper (Java) and we need a language-neutral
>>> solution for configuring the server.
>>>
>>> Some other notes:
>>> * there are some duplicated tests in hotrod-client and server
>>> integration test suites, in this case it probably makes sense to only
>>> include in the TCK the server integration test
>>> * tests from the hotrod-client module which are supposed to be part of
>>> the TCK should be copied to the server integration test suite one day
>>> (possibly later)
>>>
>>> Please let us know what you think.
>>>
>>> Thanks,
>>> Martin
>>>
>>>
>>> [1]
>>> https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_EA0giQNDZWzFNPWrF5G4/edit#gid=0
>>> [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit
>>> [3] https://github.com/infinispan/infinispan/pull/5012
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> -- 
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team