[infinispan-dev] DIST.retrieveFromRemoteSource

[Dan Berindei]

On Sat, Feb 4, 2012 at 5:23 PM, Bela Ban <bban at redhat.com> wrote:
> No, Socket.send() is not a hotspot in the Java code.
>
> On the networking level, with UDP datagrams, a packet is placed into a
> buffer and send() returns after the packet has been placed into the
> buffer successfully. Some network stacks simply discard the packet if
> the buffer is full, I assume.
>
> With TCP, send() blocks until the packet has been placed into the buffer
> and an ack has been received. If the receiver can't keep up with the
> sending rate, it'll reduce the window to 0, so send() blocks until the
> receiver catches up.
>
> So if you send 2 requests in paralle over TCP, both threads would
> block() on the send.
>
> So I don't think parallelization would make sense here, unless of course
> you're doing something else, such as serialization, lock acquisition etc...
>

You're probably right...

When I sent the email I had only seen that DataGramSocket.send takes
0.4% of the get worker's total time (which is huge considering that
99.3% is spent waiting for the remote node's response). I didn't
notice that it's synchronized and the actual sending only takes half
of that - sending the messages in parallel would create more lock
contention on the socket *and* in JGroups.

On the other hand, we're sending the messages to two different nodes,
on two different sockets, so sending in parallel *may* improve the
response time in scenarios with few worker threads. Certainly not
worth making our heavy-load performance worse, but I thought it was
worth mentioning.


Anyway, my primary motivation for this question was that I believe we
could use GroupRequest instead of our FutureCollator to send our
commands in parallel. They both do essentially the same thing, except
FutureCollator has and extra indirection layer because it uses
UnicastRequest. If there is any performance discrepancy at the moment
between GroupRequest and FutureCollator+UnicastRequest, I don't think
there is any fundamental reason why we can't "fix" GroupRequest to be
just as efficient as FutureCollator+UnicastRequest.

I think I just saw one such fix: in RequestCorrelator.sendRequest, if
anycasting is enabled then it's making a copy of the buffer for each
target member. I don't think that is necessary at all, in fact I think
it should reuse both the buffer and the headers and only change the
destination address.

Cheers
Dan


>
> On 2/4/12 3:43 PM, Manik Surtani wrote:
>> Is that a micro-optimisation?  Do we know that socket.send() really is a hotspot?
>>
>> On 1 Feb 2012, at 00:11, Dan Berindei wrote:
>>
>>> It's true, but then JGroups' GroupRequest does exactly the same thing...
>>>
>>> socket.send() takes some time too, I thought sending the request in
>>> parallel would mean calling socket.send() on a separate thread for
>>> each recipient.
>>>
>>> Cheers
>>> Dan
>>>
>>>
>>> On Fri, Jan 27, 2012 at 6:41 PM, Manik Surtani<manik at jboss.org>  wrote:
>>>> Doesn't setBlockForResults(false) mean that we're not waiting on a response, and can proceed to the next message to the next recipient?
>>>>
>>>> On 27 Jan 2012, at 16:34, Dan Berindei wrote:
>>>>
>>>>> Manik, Bela, I think we send the requests sequentially as well. In
>>>>> ReplicationTask.call:
>>>>>
>>>>>                for (Address a : targets) {
>>>>>                   NotifyingFuture<Object>  f =
>>>>> sendMessageWithFuture(constructMessage(buf, a), opts);
>>>>>                   futureCollator.watchFuture(f, a);
>>>>>                }
>>>>>
>>>>>
>>>>> In MessageDispatcher.sendMessageWithFuture:
>>>>>
>>>>>         UnicastRequest<T>  req=new UnicastRequest<T>(msg, corr, dest, options);
>>>>>         req.setBlockForResults(false);
>>>>>         req.execute();
>>>>>
>>>>>
>>>>> Did we use to send each request on a separate thread?
>>>>>
>>>>>
>>>>> Cheers
>>>>> Dan
>>>>>
>>>>>
>>>>> On Fri, Jan 27, 2012 at 1:21 PM, Bela Ban<bban at redhat.com>  wrote:
>>>>>> yes.
>>>>>>
>>>>>> On 1/27/12 12:13 PM, Manik Surtani wrote:
>>>>>>>
>>>>>>> On 25 Jan 2012, at 09:42, Bela Ban wrote:
>>>>>>>
>>>>>>>> No, parallel unicasts will be faster, as an anycast to A,B,C sends the
>>>>>>>> unicasts sequentially
>>>>>>>
>>>>>>> Is this still the case in JG 3.x?
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Bela Ban
>>>>>> Lead JGroups (http://www.jgroups.org)
>>>>>> JBoss / Red Hat
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> twitter.com/maniksurtani
>>>>
>>>> Lead, Infinispan
>>>> http://www.infinispan.org
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik at jboss.org
>> twitter.com/maniksurtani
>>
>> Lead, Infinispan
>> http://www.infinispan.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Bela Ban
> Lead JGroups (http://www.jgroups.org)
> JBoss / Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev