[infinispan-dev] Infinispan Query tests development: focus!

[Sanne Grinovero]

Hi all,
needing your advice to define the most suited level of coupling we
want of tests in Query towards Lucene API and other behavior which it
is not responsible for.

Recently the amount of tests added to Query has been significantly
reinforced; this is very welcome as it was highly needed, but I'm
getting concerned that we might getting a bit too far with it, or
actually spending energy in the wrong direction.

As most know, Infinispan Query is a module meant to integrate the
Hibernate Search "engine" module with the events and transactions from
Infinispan Core. It also happens to expose the end user to some Lucene
to define the queries.

Since this is a module integrating two other components, it must be
making some assumptions on the other two: it's not going to verify for
example if NBST behaves correctly and without blocking, but of course
tests might fail if state transfer doesn't happen at all.
Most of our tests today are end-to-end but that's not necessarily the
way to go forward.

Let's take the directory provider configuration option: FSDirectory vs
the RAMDirectory: I think it makes perfect sense to have at least one
test setting up either of them:
 - to verify the configuration properties are being applied as expected
 - to see how different nodes could interact using a shared (disk
based) directory

But I don't think it makes sense to verify that all exotic Queries we
support are going to work equally well on both: that's a waste of time
and will make our testsuite unnecessarily complex to maintain in the
long term. One directory implementation is enough to test, and
actually we might not even want to test all Lucene queries (we don't),
as long as we have some interesting examples.


Of course in the scope of the Infinispan Directory such a test makes a
lot of sense! But that's a different module, with completely different
purpose and level of test needs.

My concern is mainly related with unnecessary duplication: you are all
very welcome to contribute to Hibernate Search as much as I contribute
to Infinispan to make sure all corner cases are properly covered.

The bottomline warning: this summer we'll be moving to Lucene 4. APIs
will change significantly in the Lucene area, but only minimal changes
(hopefully) will be exposed to the Search integration API. You'd
better delegate this complexity to the isolated component as much as
possible! I can help with some 50 tests, I will not help rewriting a
thousand tests, especially if they are duplicates of other tests I've
already been working on in a different workspace.

I don't mean to suggest any hard rule. It's definitely useful to have
some working examples in the Query code base for people to read, and
to verify integration is working. Just be thoughtfull in where you
want some test to be added, and to consider if it shouldn't be better
to spend time on performance, stress tests and especially what happens
during topology changes and interactions with CacheStores configured
in different ways.. all those things which definitely are not covered
by Hibernate Search.

Cheers,
Sanne