[infinispan-dev] Transactional consistency of query

[Radim Vansa]

Hi,

while working on ISPN-7806 I am wondering how should queries work with 
transactions. Right now it seems that updates to index are done during 
either regular command execution (on originator [A]) or prepare command 
on remote nodes [B]. Both of these cause rolled-back transactions to be 
seen, so these must be treated as bugs [C].

If we index the data after committing the transaction, there would be a 
time window when we could see the updated entries but the index would 
not reflect that. That might be acceptable limitation if a 
query-matching misses some entity, but it's also possible that we 
retrieve the query result key-set and then (after retrieving full 
entities) we return something that does not match the query. One of the 
reproducers for ISPN-7806 I've written [1] triggers a situation where 
listing all Persons could return Animal (different entity type), so I 
think that there's no validity post-check (though these reproducers 
don't use transactions).

Therefore, I wonder if the index should contain only the key; maybe we 
should store an unique version and invalidate the query if some of the 
entries has changed.

If we index the data before committing the transaction, similar 
situation could happen: the index will return keys for entities that 
will match in the future but the actually returned list will contain 
stale entities.

What's the overall plan? Do we just accept inconsistencies? In that 
case, please add a verbose statement in docs and point me to that.

And if I've misinterpreted something and raised the red flag in error, 
please let me know.

Radim

[A] This seems to be a regression after moving towards async 
interceptors - our impl of 
org.hibernate.search.backend.TransactionContext is incorrectly bound to 
TransactionManager. Then we seem to be running out of transaction and 
are happy to index it right away. The thread that executes the 
interceptor handler is also dependent on ownership (due to remote 
LockCommand execution), so I think that it does not fail the local-mode 
tests.

[B] ... and it does so twice as a regression after ISPN-7840 but that's 
easy to fix.

[C] Indexing in prepare command was OK before ISPN-7840 with pessimistic 
locking which does not send the CommitCommand, but now that the QI has 
been moved below EWI it means that we're indexing before storing the 
actual values. Optimistic locking was not correct, though.

[1] 
https://github.com/rvansa/infinispan/commit/1d62c9b84888c7ac21a9811213b5657aa44ff546


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team