[infinispan-dev] Compressing Marshaller Wrapper

Thu Feb 25 09:08:15 EST 2010

The CacheStore interface is much more "clean" than the Marshaller one, for
historic reasons I suppose.
I prefer to follow the delegating CacheStore option since there are too much
entry points to Marshaller.

The use of the Marshaller interface is different in cache stores (sometimes
the buffer part / sometimes the stream part)... So I really need to
implement *all* the Marshaller interface. It may be a good idea to clean up
Marshaller ;-)

I will now subclass CloudCacheStore to add compression in order to provide a
canvas for the delegating cache store and the upcoming <compress> option
(Sorry guys but I don't plan to understand you xml configuration stuff)

phil

On Thu, Feb 25, 2010 at 2:40 PM, Philippe Van Dyck <pvdyck at gmail.com> wrote:

> Sounds like a better idea, closer to the real purpose since the Marshaller
> is used in other places... but how do I configure this ?
>
> Anyway, the marshaller is done, I am testing it, but the code is very easy
> to adapt to a cachestore...
>
> phil
>
>
> On Thu, Feb 25, 2010 at 2:33 PM, Mircea Markus <mircea.markus at jboss.com>wrote:
>
>> Can't we handle this through a delegating CacheStore, that would compress
>> the data and pass it on to the actual store for further processing (the
>> actual store will take then the binary data and marshall it (ni marshalling
>> I assume as it receives byte[]).?
>>
>> On 25 Feb 2010, at 14:06, Philippe Van Dyck wrote:
>>
>> Done
>>
>> https://jira.jboss.org/jira/browse/ISPN-357
>>
>> Phil
>>
>> On Thu, Feb 25, 2010 at 12:22 PM, Manik Surtani <manik at jboss.org> wrote:
>>
>>>
>>> On 25 Feb 2010, at 11:17, Philippe Van Dyck wrote:
>>>
>>> Thanks Manik but it is actually related to the compression of the stored
>>> and (https) transferred ones on the S3 side.
>>> The S3 bill is 95% lower here... and https transfer time is lower too
>>> (you pay for booth).
>>>
>>>
>>> Makes sense.  Create a JIRA for this, I'm sure such a feature would be
>>> generically useful to others as well - if you're happy to contribute your
>>> work back in.  :)
>>>
>>> The Marshaller interface is exposing a lot of methods and it looks like
>>> the call each other...
>>>
>>>
>>> The original Marshaller interface is legacy, from JBoss Cache, which was
>>> built as an extension to JGroups' Marshaller interface.  It may look a
>>> little confusing as a result, but have a look at the AbstractMarshaller
>>> which should help simplify things.  Further, look at the
>>> VersionAwareMarshaller, which is the entry point into the marshalling
>>> system.
>>>
>>> HTH
>>> Manik
>>>
>>>
>>>
>>>
>>> phil
>>>
>>> On Thu, Feb 25, 2010 at 12:11 PM, Manik Surtani <manik at jboss.org> wrote:
>>>
>>>> If it is compression for transmission you are concerned about, you can
>>>> do this by adding the JGroups COMPRESS protocol to your JGroups cfg.
>>>>
>>>> On 25 Feb 2010, at 11:02, philippe van dyck wrote:
>>>>
>>>> > Hi All,
>>>> >
>>>> > Currently, I compress all data before sending it to the cache. Once
>>>> compressed, I gain 95% of the JSonized qi4j objects.
>>>> >
>>>> > I did some profiling during the load tests and compression is taking
>>>> roughly 80% of the cpu time.
>>>> > So I would like to compress only the data sent to the store, not in
>>>> memory.
>>>> >
>>>> > Looks like the Marshaller is my friend here, and I plan to write a
>>>> compressing wrapper around it.
>>>> >
>>>> > Now, when I look at it, I see two ways to wrap the compression
>>>> process.
>>>> >
>>>> > One way is with the ObjectInput / ObjectOutput but I am bothered by
>>>> the reentrant flag.
>>>> > The other is the ByteBuffer stuff, no concurrency problem here, but it
>>>> looks like more work.
>>>> >
>>>> > WDYT ?
>>>> >
>>>> > Cheers,
>>>> >
>>>> > phil
>>>> > _______________________________________________
>>>> > infinispan-dev mailing list
>>>> > infinispan-dev at lists.jboss.org
>>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>>  --
>>> Manik Surtani
>>> manik at jboss.org
>>> Lead, Infinispan
>>> Lead, JBoss Cache
>>> http://www.infinispan.org
>>> http://www.jbosscache.org
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100225/3e75ff44/attachment-0002.html 