[infinispan-dev] Optimizing tx for DIST from user perspective

Mircea Markus mircea.markus at jboss.com
Mon Jul 9 03:31:57 EDT 2012


On 06/07/2012 17:12, Vladimir Blagojevic wrote:
> Mircea,
>
> Privately to you as I am not sure this makes sense for wider
> distribution - yet.
Adding infinispan-dev to this as I think it's interesting for a wider 
audience.
>
> Say user has bunch of keys/values to insert into cache. He could do it
> one key/value at the time, all in one tx or in tx batches. If he wants
> to do it in batches of transactions then it would make sense to group
> keys by the primary Address assigned on hashwheel.
Very interesting point. Besides the locking, grouping keys has another 
significant advantage : during the prepare phase each node receives the 
complete list of modifications in that transaction and not only the 
modification pertaining to it.
E.g. say we have the following key->node mapping:
k1 -> A
k2 -> B
k3 -> C
Where k1, k2 and k3 are keys; A, B and C are nodes.
If Tx1 writes (k1,k2,k3) then during the prepare A,B and C will receive 
the the same package containing all the modification - namely (k1, 
k2,k3). There are several reasons for doing this (apparently) 
unoptimized approach: serialize the prepare only once, better handling 
of recovery information.

Now if you group transactions/batches base on key distribution, as you 
suggested, the amount of redundant traffic is significantly reduced - 
and that translates in better performance especially when the datasets 
you're inserting is quite high.
> Therefore each tx batch would lock keys only on primary node and
> nowhere else - call it tx node pinning if you want! Now imagine a
> cluster with bunch of concurrent txs initiated from all nodes. If I am
> not mistaken this tx pinning algorithm would not only increase
> throughput but also minimize deadlocks.
yes. With optimistic tx caches, the only possibility for deadlocks is 
between transactions touching multiple nodes[1]. As long as your 
transactions only write to the same node, even if they do it on the same 
key-set, the possibility of deadlock is (almost[2]) zero.


> Does this make sense? If so, why not support it somehow on API level
> or do we already? ;-)
We don't have a service like this for now. I think your best option is 
to fetch the CH from the advanced cache 
(cache.getAdvancedCache().getDistributionManager().getConistentHahs()) 
and use it to group the inserts.

Thinking about it, might be worth having an blog entry describing this 
as it can really boost performance when you need to load an initial 
large set of data in Infinispan.

> Regards,
> Vladimir


[1] this DLD situation will be fixed once we have incremental locking in 
place: https://issues.jboss.org/browse/ISPN-1219
[2]  we use key's CH value to induce an order over the keys written in a 
transaction - that's in order to avoid deadlocks. If there are 
collisions between these values then there's still a chance for deadlock.



More information about the infinispan-dev mailing list