[infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

[Bela Ban]

On 5/23/11 11:09 PM, Dan Berindei wrote:

>> No need to expose the ExposedByteArrayOutputStream, a byte[] buffer,
>> offset and length will do it, and we already use this today.
>>
>>
>>> In case the value is not stored in binary form, the expected life of
>>> the stream is very short anyway, after being pushed directly to
>>> network buffers we don't need it anymore... couldn't we pass the
>>> non-truncated stream directly to JGroups without this final size
>>> adjustement ?
>>
>
> The problem is that byte[] first has to be copied to another buffer
> together with the rest of the ReplicableCommand before getting to
> JGroups. AFAIK in JGroups you must have 1 buffer for each message.


If you use ExposedByteArrayOutputStream, you should have access to the 
underlying buffer, so you don't need to copy it.


>> You do that, yes.
>>
>> However, afair, the issue is not on the *sending*, but on the
>> *receiving* side. That's where the larger-than-needed buffer sticks
>> around. On the sending side, as you mentioned, Infinispan passes a
>> buffer/offset/length to JGroups and JGroups passes this right on to the
>> network layer, which copies that data into a buffer.
>>
>
> I don't think so... on the receiving size the buffer size is
> controlled exclusively by JGroups, the unmarshaller doesn't create any
> buffers. The only buffers on the receiving side are those created by
> JGroups, and JGroups knows the message size before creating the buffer
> so it doesn't have to worry about predicting buffer sizes.
>
> On sending however I understood that JGroups keeps the buffer with the
> offset and length in the NakReceivingWindow exactly as it got it from
> Infinispan, without any trimming, until it receives a STABLE message
> from all the other nodes in the cluster.


Ah, ok. I think we should really do what we said before JBW, namely have 
an interactive debugging session, to clear this up.

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss