[infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

[Bela Ban]

On 5/23/11 8:42 PM, Dan Berindei wrote:
> On Mon, May 23, 2011 at 7:44 PM, Sanne Grinovero
> <sanne.grinovero at gmail.com>  wrote:
>> To keep stuff simple, I'd add an alternative feature instead:
>> have the custom externalizers to optionally recommend an allocation buffer size.
>>
>> In my experience people use a set of well known types for the key, and
>> maybe for the value as well, for which they actually know the output
>> byte size, so there's no point in Infinispan to try guessing the size
>> and then adapting on it; an exception being the often used Strings,
>> even in composite keys, but again as user of the API I have a pretty
>> good idea of the size I'm going to need, for each object I store.
>>
>
> Excellent idea, if the custom externalizer can give us the exact size
> of the serialized object we wouldn't need to do any guesswork.
> I'm a bit worried about over-zealous externalizers that will spend
> just as much computing the size of a complex object graph as they
> spend on actually serializing the whole thing, but as long as our
> internal externalizers are good examples I think we're ok.
>
> Big plus: we could use the size of the serialized object to estimate
> the memory usage of each cache entry, so maybe with this we could
> finally constrain the cache to use a fixed amount of memory :)


I don't think this is a good idea because most people won't be able to 
guess the right buffer sizes. Giving inncorrect buffer sizes might even 
lead to performance degradation, until the buffers have expanded...

For example, would you guys be able to guess the buffer sizes of 
Infinispan used in JBoss AS ? We're placing not just session data, but 
all sorts of crap into the cache, so I for one wouldn't be able to even 
give you a best estimate...

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss