[infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

Bela Ban bban at redhat.com
Mon May 23 16:50:36 EDT 2011



On 5/23/11 6:44 PM, Sanne Grinovero wrote:
> To keep stuff simple, I'd add an alternative feature instead:
> have the custom externalizers to optionally recommend an allocation buffer size.
>
> In my experience people use a set of well known types for the key, and
> maybe for the value as well, for which they actually know the output
> byte size, so there's no point in Infinispan to try guessing the size
> and then adapting on it; an exception being the often used Strings,
> even in composite keys, but again as user of the API I have a pretty
> good idea of the size I'm going to need, for each object I store.
>
> Also in MarshalledValue I see that an ExposedByteArrayOutputStream is
> created, and after serialization if the buffer is found to be bigger
> than the buffer we're referencing a copy is made to create an exact
> matching byte[].
> What about revamping the interface there, to expose the
> ExposedByteArrayOutputStream instead of byte[], up to the JGroups
> level?


No need to expose the ExposedByteArrayOutputStream, a byte[] buffer, 
offset and length will do it, and we already use this today.


> In case the value is not stored in binary form, the expected life of
> the stream is very short anyway, after being pushed directly to
> network buffers we don't need it anymore... couldn't we pass the
> non-truncated stream directly to JGroups without this final size
> adjustement ?


You do that, yes.

However, afair, the issue is not on the *sending*, but on the 
*receiving* side. That's where the larger-than-needed buffer sticks 
around. On the sending side, as you mentioned, Infinispan passes a 
buffer/offset/length to JGroups and JGroups passes this right on to the 
network layer, which copies that data into a buffer.

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss


More information about the infinispan-dev mailing list