[infinispan-dev] Major version cleaning
Tristan Tarrant
ttarrant at redhat.com
Tue Feb 21 11:16:37 EST 2017
On 21/02/17 16:29, Sanne Grinovero wrote:
>> You haven't explained what "flush" means. Since you separate that from
>> atomicity/consistency, I assume that batches on non-tx cache are just
>> ordered putOrRemoveAll operations, immediately visible on flush without
>> any atomicity.
I assume that in Sanne's idea, ordering in a batch doesn't matter, aside
from operations on the same key. Having ordering in there would for
example not allow us to parallelize by segment.
> So I want to write a first chunk, in our code that looks like:
>
> startBatch
> put(chunk1/A, [some large value])
> put(chunk1/B, [some small metadata])
> put(chunk1/C, [some small metadata])
> endBatch
> There is no reason to use a transaction, in fact we had to disable
> transactions as some of these entries could be large.
> There also is no reason for the batch, other than optimising the latency.
Let me summarize to see if we have the requirements for a useful
batching system (which is sort of patterned on the JDBC statement batching):
- a batch is not an atomic operation, i.e. it is not backed by a transaction
- it can be wrapped in a transaction if needed
- batches cannot be nested
- batches only involve unconditional write operations (put, putAll, remove)
- ordering of operations within a batch is unimportant aside from
modifications to the same key where we apply "last one wins"
- when a batch is "flushed" (i.e. endBatch is invoked) the ops are
grouped by segment and sent to the appropriate owner for processing,
potentially in parallel
As Radim has called it, this is essentially a putOrRemoveAll op (with an
async counterpart).
Is that summary correct ?
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
More information about the infinispan-dev
mailing list