[infinispan-dev] Removing batching (transactions) requirement from the Lucene Directory

[Sanne Grinovero]

Context reminder: the Infinispan Lucene Directory requires batching.

The reason it's using batching was mostly to find a tradeoff between
not too big blob sizes (chunking of larger blocks) and reduce network
related blocking invocations, taking advantage of the fact we do know
pretty well how Lucene will drive this custom structure and how to use
Infinispan.. still the amount of other optimizations we applied over
time made actual usage of batching not a frequent operation any more,
and I see now some reasons to remove batch usage altogether:

- batching requires transactions -> since we can't use some
non-transactional operations on a transactional able cache any more,
this was enabling auto-commit on the many operations which should not
use transactions, and negatively impacting performance.

- by using batching (transactions) internally we can't have it
participate in an existing transaction (without the perils of nested
transactions).

- for those cases transactions are not desired, it's slowing down all
operations since a cache is now either fully transactional or not at
all.

By running some simple performance tests it even seems that removing
these batch operations actually improved performance a little bit;
this might need some more investigation and a better test, but at
least it makes me think that performance is not going to drop
noticeably if we remove it.

WDYT?

Sanne