[infinispan-dev] Minutes Infinispan meeting in London

Sanne Grinovero sanne at infinispan.org
Mon Mar 4 05:43:50 EST 2013


Thanks Bela,
I've moved it to the wiki:
https://community.jboss.org/wiki/ClusteringMeetingLondonFeb2013

Sanne

On 24 February 2013 10:26, Bela Ban <bban at redhat.com> wrote:
>
> Mircea, Dan, Pedro, Sanne and I had a meeting in London this week on how
> to use the new features of JGroups 3.3 in Infinispan 5.3, I've copied
> the minutes from the wiki below.
>
>
>   London meeting
>
> Bela, Pedro, Mircea, Dan, Sanne
>
>
>       Message bundling and OOB messages
>
> In 3.3, all messages will be bundled, not just regular messages, but
>
> also OOB messages. The way this works on the sender side is:
>
> - A thread sending a message in the transport adds it to a queue
>
> - There's one thread which dequeues messages and sends them as bundles
>
>    - It sends a messages bundle if the max size has been reached, or
>
>      there are no more messages in the queue
>
>    - This means single messages are sent immediately, or we fill up a
>
>      bundle (in a few microseconds) and send it
>
> Impact on Infinispan:
>
> - Use DONT_BUNDLE instead of OOB if you don't want to bundle messages
>
>    - However, even DONT_BUNDLE might get deprecated
>
> - If we have 1 sender invoking sync RPCs, we don't need to set
>
>    DONT_BUNDLE anymore
>
> - If we have multiple senders invoking sync RPCs, performance should
>
>    get better as RPCs and responses are bundled
>
> - Since bundling will result in message *batches* on the receiver,
>
>    performance should increase in general
>
>
>       Message batching
>
> Message bundles sent by a sender are received as message batches
>
> (MessageBatch) by the receivers. When a batch is received, the batch
>
> is passed up using up(MessageBatch).
>
> Protocols can remove / replace / add messages in a batch and pass the
>
> batch further up.
>
> The advantage of a batch is that resources such as locks are acquired
>
> only once for a batch of N messages rather than N times. Example: when
>
> NAKACK2 receives a batch of 10 messages, it adds the 10 messages to
>
> the receiver table in a bulk operation, which is more efficient than
>
> doing this 10 times.
>
> Further optimizations on batching (probably 3.4):
>
> - Remove similar ops, e.g. UNICAST3 acks for A:15, A:25 and A:35 can
>
>    be clubbed together into just ack(A:35)
>
> - Merge similar headers, e.g. multicast messages 20-30 can be orderd
>
>    by seqno, and we simply send a range [20..30] and let the receiver
>
>    generate the headers on the fly
>
>
>       Async Invocation API (AIA)
>
> JGroups only passes up messages to Infinispan, which then uses its own
>
> thread pool to deliver them. E.g. based on Pedro's code for TO, we
>
> could parallelize delivery based on the target keys of the
>
> transaction. E.g if we have tx1 modifying keys {A,B,C} and tx2
>
> modifying keys {T,U}, then tx1 and tx2 can be run concurrently.
>
> If tx1 and tx2 modify overlapping key sets, then tx2 would be queued
>
> and executed *after* tx1, not taking up a thread from the pool,
>
> reducing the chances of the thread pool maxing out and also
>
> ensuring different threads are not going to contend on the locks
>
> on same keys.
>
> The implementation could be done in an interceptor fronting the
> interceptor stack, which queues dependent TXs and
>
> - when ready to be executed - sends them up the interceptor stack on a
> thread from
>
> the internal pool.
>
> Infinispan having its own thread pool means that JGroups threads will
>
> not block anymore, e.g. trying to acquire a lock for a TX. The size of
>
> those pools can therefore be reduced.
>
> The advantage of AIA is that it's up to Infinispan, not JGroups, how
>
> to deliver messages. JGroups delivers messages based on the order in
>
> which they were sent by a sender (FIFO), whereas Infinispan can make
>
> much more informed decisions as to how to deliver the messages.
>
>
>       Internal thread pool for JGroups
>
> All JGroups internal messages use the internal thread pool (message
>
> flag=INTERNAL). Not having to share the OOB pool with apps (such as
>
> Infinispan) means that internal messages can always be processed, and
>
> are not discarded or blocked, e.g. by a maxed out thread pool.
>
> The internal pool can be switched off, and - if AIA is implemented in
>
> Infinispan - the number of OOB and regular threads can be massively
>
> reduced. The internal thread pool doesn't need to be big either.
>
>
>       UNICAST3
>
> Successor to UNICAST and UNICAST2, best of both worlds. Acks single
>
> messages quickly, so we have no first-msg-lost or last-msg-lost issues
>
> anymore. Doesn't generate many acks though.
>
> Proposed to trigger and ACK only after a certain number of messages
>
> rather than after any batch to avoid ACK on small batches.
>
> https://issues.jboss.org/browse/JGRP-1594
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


More information about the infinispan-dev mailing list