[hibernate-dev] Search: paging over deleted entries [ISPN-2206]

[Sanne Grinovero]

Hi all,
I feel the need to share some thoughts on this quite tricky patch proposal:
https://github.com/infinispan/infinispan/pull/1245

I'm tempted to say that Hibernate Search should "scan ahead" to look
for more results to fill the gap; but -even assuming this was easy to
implement (which it is not)- this behaviour would be inconsistent with
update operations, or even inserts.

For inserts we could compensate by keeping an in-memory index paired
to the current transactions, and consider this additional index as a
temporary additional shard; by following this path I'm confident we
could also implement proper removals and updates using a custom collector,
but this will definitely be more complex and introduce some overhead.

Overhead could be minimized by considering this temporary in-memory
index as a pre-analysed dataset, so that we avoid doing the work again
at commit time.

Any opinions on how this should work?

Cheers,
Sanne