Re: [infinispan-dev] storing in memory data in binary format

LazyMarshalling (use of marshalled values internally) does precisely this. If MarshalledValues are used, then calculating the size is quite easy, except that it may always spike to 2x the byte[] when both forms (serialized and unserialized) are used. This is a spike though and one form is always cleared out at the end of every invocation.

On 2 Oct 2009, at 12:44, Mircea Markus wrote: > Hi, > > While working on the coherence config converter, I've seen that they > are able to specify how much memory a cache should use at a time (we > also had a thought about this in past but abandoned the idea mainly > due to performance). E.g. : > <backing-map-scheme> > <local-scheme> > <high-units>100m</high-units> > <unit-calculator>BINARY</unit-calculator> > <eviction-policy>LRU</eviction-policy> > </local-scheme> > </backing-map-scheme> > When 100 MB is reached, data will start to be evicted. I know we > support marshaled values, but it's not exactly the same thing. > I've been wondering how do they achieve this, and I've found this http://coherence.oracle.com/display/COH35UG/Storage+and+Backing+Map > . > The way it works (my understanding!) is by keeping the key + values > in > serialized form within the map. So, if you have both the key and > value > as a byte[], you can easily measure the memory fingerprint of the > cached data. > Now what about keeping data in maps in serialized form? > Pros: > - we would be able to support memory based eviction triggers. > - in DIST mode, when doing a put we won't need to deserialize the > data > at the other end. This de-serialization might be redundant, as if > another node asks for this data, we'll have to serialize it back etc. > - the sync puts would be faster, as the data gets only serialized > (and > doesn't get deserialized at the other end). > - ??? > Cons: > - data would be deserialized for each get request, adding a latency. > Partially compensated by faster puts (see cons) and can be mitigated > by using L1 caches (near caches) > - ??? > > Well I'm not even sure that this fits with our actual architecture, > just brought this in for brainstorming :) > > Cheers, > Mircea > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev