[infinispan-dev] HotRod client TCK

Wed May 10 04:09:28 EDT 2017

On Mon, May 8, 2017 at 12:10 PM, Galder Zamarreño <galder at redhat.com> wrote:

> I think there's some value in Radim's suggestion. The email was not fully
> clear to me initially but after reading a few times I understood what he
> was referring to. @Radim, correct me if I'm wrong...
>
> Right now clients verify that they behave as expected, e.g. JS client uses
> its asserts, Java client uses other asserts. What Radim is trying to say is
> that there needs to be a way to verify they work adequately independent of
> their implementations.
>
> So, the only way to do that is to verify it at the server level.
>
Not sure what exactly he means by the fake server, but more than a fake
> server, I'd be more inclined to modify the server to that it can somehow
> act as TCK verifier.

We had a thread about Hot Rod testing last year, and another possible
strategy is to use real unmodified servers have the TCK written once in a
neutral language, compile/distribute it to each of the clients, where the
tests would run as part of the build. More details on [1]

[1]
http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Hot-Rod-testing-tt4031152.html#a4031213

Gustavo

> This is to avoid having to reimplement transport logic, protocol
> decoder...etc in a new fake server.
>
> Cheers,
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
> > On 11 Apr 2017, at 15:57, Radim Vansa <rvansa at redhat.com> wrote:
> >
> > Since these tests use real server(s), many of them test not only the
> > client behaviour (generating correct commands according to the
> > protocol), but server, too. While this is practical (we need to test
> > server somehow, too), there's nothing all the tests across languages
> > will have physically in common and all comparison is prone to human
> error.
> >
> > If we want to test various implementations of the client, maybe it would
> > make sense to give the clients a fake server that will have just a
> > scenario of expected commands to receive and pre-defined responses. We
> > could use audit log to generate such scenario based on the actual Java
> > tests.
> >
> > But then we'd have to test the actual behaviour on server, and we'd need
> > a way to issue the commands.
> >
> > Just my 2c
> >
> > Radim
> >
> > On 04/11/2017 02:33 PM, Martin Gencur wrote:
> >> Hello all,
> >> we have been working on https://issues.jboss.org/browse/ISPN-7120.
> >>
> >> Anna has finished the first step from the JIRA - collecting information
> >> about tests in the Java HotRod client test suite (including server
> >> integration tests) and it is now prepared for wider review.
> >>
> >> She created a spreadsheet [1]. The spread sheet includes for each Java
> >> test its name, the suggested target package in the TCK, whether to
> >> include it in the TCK or not, and some other notes. The suggested
> >> package also poses grouping for the tests (e.g. tck.query, tck.near,
> >> tck.xsite, ...)
> >>
> >> Let me add that right now the goal is not to create a true TCK [2]. The
> >> goal is to make sure that all implementations of the HotRod protocol
> >> have sufficient test coverage and possibly the same server side of the
> >> client-server test (including the server version and configuration).
> >>
> >> What are the next step?
> >>
> >> * Please review the list (at least a quick look) and see if some of the
> >> tests which are NOT suggested for the TCK should be added or vice versa.
> >> * I suppose the next step would then be to check other implementations
> >> (C#, C++, NodeJS, ..) and identify tests which are missing there (there
> >> will surely be some).
> >> * Gradually implement the missing tests in the other implementations
> >>    Note: Here we should ensure that the server is configured in the same
> >> way for all implementations. One way to achieve this (thanks Anna for
> >> suggestion!) is to have a shell/batch scripts for CLI which would be
> >> executed before the tests. This can probably be done for all impls. and
> >> both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes
> >> useless because it uses Creaper (Java) and we need a language-neutral
> >> solution for configuring the server.
> >>
> >> Some other notes:
> >> * there are some duplicated tests in hotrod-client and server
> >> integration test suites, in this case it probably makes sense to only
> >> include in the TCK the server integration test
> >> * tests from the hotrod-client module which are supposed to be part of
> >> the TCK should be copied to the server integration test suite one day
> >> (possibly later)
> >>
> >> Please let us know what you think.
> >>
> >> Thanks,
> >> Martin
> >>
> >>
> >> [1]
> >> https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_
> EA0giQNDZWzFNPWrF5G4/edit#gid=0
> >> [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit
> >> [3] https://github.com/infinispan/infinispan/pull/5012
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > --
> > Radim Vansa <rvansa at redhat.com>
> > JBoss Performance Team
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170510/2eb6619b/attachment-0001.html 