[infinispan-dev] HotRod client - optimize serialization

[Mircea Markus]

On 5 Jul 2010, at 15:48, Galder Zamarreño wrote:

> 
> On Jul 5, 2010, at 12:10 PM, Mircea Markus wrote:
> 
>> Hi,
>> 
>> On HR we serialize the (key, value) pairs before sending them to the client.
>> Current approach is we transfer them into an byte[] using an ByteArrayInputStream/ByteArrayOutputStream and then write them over the network with socket.getOutputStream().write(byte[])  
>> 
>> What I'm looking for a better way of serializing, by reusing byte arrays.
>> One approach would be to use an pool of ExposedByteArrayOutputStream, pool's size being == number of tcp connections between client and server. My concern with this approach is that if one is using a large value (e.g. 100MB ) once in a blue moon, than I'll always keep an 100MB array in memory, cached, even though I don't want it. 
> 
> Hmmm, why do you want to have a pool? When replicating/distributing stuff, we're simply creating an ExposedByteArrayOutputStream with initial size of 128 everytime we need to send something across. I did do a fair bit of profiling last year of the marshalling layer and creating EBAOS instances was not expensive at all. What was expensive though was creating OOS instances, which is why we used JBoss Marshalling that allowed us to pool those.
Thanks for input - I won't pool :)
> 
>> 
>> Another approach would be to use use existing code for keys(i.e. serialize them into an byte[]), which are expected to be smaller, and for values to write directly in the  socket, through socket.getOutputStream(). This way I won't have the 100MB issue and also I won't create an byte[] for each value (I need to do that for keys though, as I need access to key's byet[] for computing its hash code).
>> 
>> Any suggestions much appreciated!
>> 
>> Cheers,
>> Mircea
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev