4:24 a.m.
On 2/5/12 9:44 AM, Dan Berindei wrote:
On Sat, Feb 4, 2012 at 5:23 PM, Bela Ban<bban(a)redhat.com>
wrote:
> No, Socket.send() is not a hotspot in the Java code.
>
> On the networking level, with UDP datagrams, a packet is placed into a
> buffer and send() returns after the packet has been placed into the
> buffer successfully. Some network stacks simply discard the packet if
> the buffer is full, I assume.
>
> With TCP, send() blocks until the packet has been placed into the buffer
> and an ack has been received. If the receiver can't keep up with the
> sending rate, it'll reduce the window to 0, so send() blocks until the
> receiver catches up.
>
> So if you send 2 requests in paralle over TCP, both threads would
> block() on the send.
>
> So I don't think parallelization would make sense here, unless of course
> you're doing something else, such as serialization, lock acquisition etc...
>
You're probably right...
When I sent the email I had only seen that DataGramSocket.send takes
0.4% of the get worker's total time (which is huge considering that
99.3% is spent waiting for the remote node's response). I didn't
notice that it's synchronized and the actual sending only takes half
of that - sending the messages in parallel would create more lock
contention on the socket *and* in JGroups.
On the other hand, we're sending the messages to two different nodes,
on two different sockets, so sending in parallel *may* improve the
response time in scenarios with few worker threads. Certainly not
worth making our heavy-load performance worse, but I thought it was
worth mentioning.
If you think this makes a difference, why don't you make a temporary
code change and measure its effect on performance ?
Anyway, my primary motivation for this question was that I believe
we
could use GroupRequest instead of our FutureCollator to send our
commands in parallel. They both do essentially the same thing, except
FutureCollator has and extra indirection layer because it uses
UnicastRequest.
Ah, ok. Yes, I think that 1 GroupRequest of 2 might be more efficient
than 2 UnicastRequests for sending an 'anycast' message to 2 members.
Perhaps the marshalling is done only once (I'd have to confirm that
though) and we're only creating 1 data structure and add it to a hashmap
(request/response correlation)... Certainly worth giving a try...
If there is any performance discrepancy at the moment
between GroupRequest and FutureCollator+UnicastRequest, I don't think
there is any fundamental reason why we can't "fix" GroupRequest to be
just as efficient as FutureCollator+UnicastRequest.
You're assuming that FutureCollator+UnicastRequest is faster than
GroupRequest, what are you basing your assumption on ? As I outlined
above, I'd rather assume the opposite, although I don't know FutureCollator.
Maybe we should have a chat next week to go through the code that sends
an anycast to 2 members, wdyt ?
I think I just saw one such fix: in RequestCorrelator.sendRequest,
if
anycasting is enabled then it's making a copy of the buffer for each
target member. I don't think that is necessary at all, in fact I think
it should reuse both the buffer and the headers and only change the
destination address.
No, this *is* necessary, I don't just make copies because I think this
is fun !! :-)
Remember that a message might be retransmitted, so it is placed into a
retransmit buffer. If M1 has destination A and M2 has destination B, and
we send M1 first (to A), then change M1's destination to B, and send it,
everything is fine. However, if we later get a retransmit request from
B, we'd resend the message to A instead ! This is just 1 example,
modifications of headers is another one.
Note that the copy does *not* copy the buffer (payload) itself, but only
references it, so this is fast. Of course, nobody is supposed to modify
the contents of the buffer itself...
--
Bela Ban
Lead JGroups (http://www.jgroups.org)
JBoss / Red Hat