[infinispan-dev] Infinispan Query tests development: focus!

Hi all, needing your advice to define the most suited level of coupling we want of tests in Query towards Lucene API and other behavior which it is not responsible for. Recently the amount of tests added to Query has been significantly reinforced; this is very welcome as it was highly needed, but I'm getting concerned that we might getting a bit too far with it, or actually spending energy in the wrong direction. As most know, Infinispan Query is a module meant to integrate the Hibernate Search "engine" module with the events and transactions from Infinispan Core. It also happens to expose the end user to some Lucene to define the queries. Since this is a module integrating two other components, it must be making some assumptions on the other two: it's not going to verify for example if NBST behaves correctly and without blocking, but of course tests might fail if state transfer doesn't happen at all. Most of our tests today are end-to-end but that's not necessarily the way to go forward. Let's take the directory provider configuration option: FSDirectory vs the RAMDirectory: I think it makes perfect sense to have at least one test setting up either of them: - to verify the configuration properties are being applied as expected - to see how different nodes could interact using a shared (disk based) directory But I don't think it makes sense to verify that all exotic Queries we support are going to work equally well on both: that's a waste of time and will make our testsuite unnecessarily complex to maintain in the long term. One directory implementation is enough to test, and actually we might not even want to test all Lucene queries (we don't), as long as we have some interesting examples. Of course in the scope of the Infinispan Directory such a test makes a lot of sense! But that's a different module, with completely different purpose and level of test needs. My concern is mainly related with unnecessary duplication: you are all very welcome to contribute to Hibernate Search as much as I contribute to Infinispan to make sure all corner cases are properly covered. The bottomline warning: this summer we'll be moving to Lucene 4. APIs will change significantly in the Lucene area, but only minimal changes (hopefully) will be exposed to the Search integration API. You'd better delegate this complexity to the isolated component as much as possible! I can help with some 50 tests, I will not help rewriting a thousand tests, especially if they are duplicates of other tests I've already been working on in a different workspace. I don't mean to suggest any hard rule. It's definitely useful to have some working examples in the Query code base for people to read, and to verify integration is working. Just be thoughtfull in where you want some test to be added, and to consider if it shouldn't be better to spend time on performance, stress tests and especially what happens during topology changes and interactions with CacheStores configured in different ways.. all those things which definitely are not covered by Hibernate Search. Cheers, Sanne