[hibernate-dev] Search: paging over deleted entries [ISPN-2206]

Hi all, I feel the need to share some thoughts on this quite tricky patch proposal: https://github.com/infinispan/infinispan/pull/1245 I'm tempted to say that Hibernate Search should "scan ahead" to look for more results to fill the gap; but -even assuming this was easy to implement (which it is not)- this behaviour would be inconsistent with update operations, or even inserts. For inserts we could compensate by keeping an in-memory index paired to the current transactions, and consider this additional index as a temporary additional shard; by following this path I'm confident we could also implement proper removals and updates using a custom collector, but this will definitely be more complex and introduce some overhead. Overhead could be minimized by considering this temporary in-memory index as a pre-analysed dataset, so that we avoid doing the work again at commit time. Any opinions on how this should work? Cheers, Sanne