[infinispan-dev] Infinispan Query tests development: focus!

[Mircea Markus]

Does this have to do with the queries submitted by QA? If so it's worth having a chat with Martin on how this functionality is tested and focus it on the integration points.
+1 for not adding tests that are testing other products (lucene, hsearch) so please filter out this code from the pull request.

On 23 Apr 2013, at 12:46, Sanne Grinovero wrote:
> Hi all,
> needing your advice to define the most suited level of coupling we
> want of tests in Query towards Lucene API and other behavior which it
> is not responsible for.
> 
> Recently the amount of tests added to Query has been significantly
> reinforced; this is very welcome as it was highly needed, but I'm
> getting concerned that we might getting a bit too far with it, or
> actually spending energy in the wrong direction.
> 
> As most know, Infinispan Query is a module meant to integrate the
> Hibernate Search "engine" module with the events and transactions from
> Infinispan Core. It also happens to expose the end user to some Lucene
> to define the queries.
> 
> Since this is a module integrating two other components, it must be
> making some assumptions on the other two: it's not going to verify for
> example if NBST behaves correctly and without blocking, but of course
> tests might fail if state transfer doesn't happen at all.
> Most of our tests today are end-to-end but that's not necessarily the
> way to go forward.
> 
> Let's take the directory provider configuration option: FSDirectory vs
> the RAMDirectory: I think it makes perfect sense to have at least one
> test setting up either of them:
> - to verify the configuration properties are being applied as expected
> - to see how different nodes could interact using a shared (disk
> based) directory
> 
> But I don't think it makes sense to verify that all exotic Queries we
> support are going to work equally well on both: that's a waste of time
> and will make our testsuite unnecessarily complex to maintain in the
> long term. One directory implementation is enough to test, and
> actually we might not even want to test all Lucene queries (we don't),
> as long as we have some interesting examples.
> 
> 
> Of course in the scope of the Infinispan Directory such a test makes a
> lot of sense! But that's a different module, with completely different
> purpose and level of test needs.
> 
> My concern is mainly related with unnecessary duplication: you are all
> very welcome to contribute to Hibernate Search as much as I contribute
> to Infinispan to make sure all corner cases are properly covered.
> 
> The bottomline warning: this summer we'll be moving to Lucene 4. APIs
> will change significantly in the Lucene area, but only minimal changes
> (hopefully) will be exposed to the Search integration API. You'd
> better delegate this complexity to the isolated component as much as
> possible! I can help with some 50 tests, I will not help rewriting a
> thousand tests, especially if they are duplicates of other tests I've
> already been working on in a different workspace.
> 
> I don't mean to suggest any hard rule. It's definitely useful to have
> some working examples in the Query code base for people to read, and
> to verify integration is working. Just be thoughtfull in where you
> want some test to be added, and to consider if it shouldn't be better
> to spend time on performance, stress tests and especially what happens
> during topology changes and interactions with CacheStores configured
> in different ways.. all those things which definitely are not covered
> by Hibernate Search.
> 
> Cheers,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)