[infinispan-dev] DIST.retrieveFromRemoteSource

[Sanne Grinovero]

[cut]
>> I agree, we should not ask all replicas for the same information.
>> Asking only one is the opposite though: I think this should be a
>> configuration option to ask for any value between (1 and numOwner).
>> That's because I understand it might be beneficial to ask to more than
>> one node immediately,
> why is it more beneficial to ask multiple members than a single one? I guess it doesn't have to do with consistency, as in that case it would be required (vs beneficial).
> Is it because one of the nodes might reply faster? I'm not that sure that compensates the burden of numOwner-1 additional RPCs, but a benchmark will tell us just that.

One node might be busy doing GC and stay unresponsive for a whole
second or longer, another one might be actually crashed and you didn't
know that yet, these are unlikely but possible.
More likely, a rehash is in progress, you could then be asking a node
which doesn't yet (or anymore) have the value.

All good reasons for which imho it makes sense to send out "a couple"
of requests in parallel, but I'd unlikely want to send more than 2,
and I agree often 1 might be enough.
Maybe it should even optimize for the most common case: send out just
one, have a more aggressive timeout and in case of trouble ask for the
next node.

In addition, sending a single request might spare us some Future,
await+notify messing in terms of CPU cost of sending the request.