[infinispan-dev] The "Triangle" pattern for reducing Put latency

[Radim Vansa]

On 11/25/2015 12:07 PM, Sanne Grinovero wrote:
> On 25 November 2015 at 10:48, Pedro Ruivo <pedro at infinispan.org> wrote:
>> Comment inline.
>>
>> On 11/25/2015 10:27 AM, Sanne Grinovero wrote:
>>> At our last face to face meeting we were discussing about possible
>>> strategies to reduce the latency of a Put operation.
>>>
>>> I wasn't part of that conversation, but see in our notes that the idea
>>> would be to have the Originator to wait for first ACK from a backup,
>>> rather than having the backups confirm with the Owner first, although
>>> this would need for Gets to have to go to the primary owner
>>> exclusively to keep the current consistency semantics.
>>>
>>> An alternative is to wait for all ACKs, but I think this could still
>>> be optimised in "triangle shape" too by having the Originator only
>>> wait for the ACKs from the non-primary replicas?
>>> So backup owners have to send a confirmation message to the
>>> Originator, while the Primary owner isn't expecting to do so.
>> IMO, we should wait for all ACKs to keep our read design.

What exactly is our 'read design'?

I think that the source of optimization is that once primary decides to 
backup the operation, he can forget about it and unlock the entry. So, 
we don't need any ACK from primary unless it's an exception/noop 
notification (as with conditional ops). If primary waited for ACK from 
backup, we wouldn't save anything.

The gains are:
* less hops (3 instead of 4 if O != P && O != B)
* less messages (primary ACK is transitive based on ack from B)
* shorter lock times (not locking during P -> B RPC)

>> However, the
>> Originator needs to wait for the ACK from Primary because of conditional
>> operations and functional API.
> If the operation is successful, Primary will have to let the
> secondaries know so these can reply to the Originator directly: still
> saves an hop.
>
>> In this first case, if the conditional operation fail, the Backups are
>> not bothered. The latter case, we may need the return value from the
>> function.
> Right, for a failed or rejected operation the secondaries won't even
> know about it,
> so the Primary is in charge of letting the Originator know.
> Essentially you're highlighting that the Originator needs to wait for
> either the response from secondaries (all of them?)
> or from the Primary.
>
>>> I suspect the tricky part is what happens when the Primary owner rules
>>> +1 to apply the change, but then the backup owners (all or some of
>>> them) somehow fail before letting the Originator know. The Originator
>>> in this case should seek confirmation about its operation state
>>> (success?) with the Primary owner; this implies that the Primary owner
>>> needs to keep track of what it's applied and track failures too, and
>>> this log needs to be pruned.

Currently, in case of lost (timed out) ACK from B to P, we just report 
exception and don't care about synchronizing P and B - B can already 
store updated value.
So we don't have to care about rollback on P if replication to B fails 
either - we just report that it's broken, sorry.
Better consolidation API would be nice, though, something like 
cache.getAllVersions().

Radim


>>>
>>> Sounds pretty nice, or am I missing other difficulties?
>>>
>>> Thanks,
>>> Sanne
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team