Re: [infinispan-dev] Hot Rod testing

I also like this idea for a Unit-Based TCK for all clients, if this is possible. > - we identify and group the tests depending on their scope (basic > protocol ops, bulk ops, topology/failover, security, etc). A client > which implements the functionality of a group MUST pass all of the tests > in that group with NO exceptions This makes sense to me, but I also agree that the hard part will be in categorizing the tests into these buckets. Should the groups be divided by intelligence as well? I'm just wondering about "dumb" clients like REST and Memcached. > - we assign a unique identifier to each group/test combination (e.g. > HR.BASIC.PUT, HR.BASIC.PUT_FLAGS_SKIP_LOAD, etc). These should be > collected in a "test book" (some kind of structured file) for comparison > with client test runs Are these identifiers just used as the JUNit test group names? > - we refactor the Java client testsuite according to the above grouping > / naming strategy so that testsuite which use the wrapping approach > (i.e. C++ with SWIG) can consume it by directly specifying the supported > groups This makes sense to me as well. I think the other requirements here are that the client tests must use a real server distribution and not the embedded server. Any non-duplicated tests from the server integration test suite have to be migrated to the client test suite as well. I think this also is an opportunity to inventory the client test suite and reduce it to the most minimal number of tests that verify the adherence to the protocol and expected behavior beyond the protocol.