[infinispan-dev] Compressing Marshaller Wrapper

Thu Feb 25 09:17:11 EST 2010

Another solution is to include a "compression flag" in the version itself.

WDYT ?

phil

On Thu, Feb 25, 2010 at 3:15 PM, Philippe Van Dyck <pvdyck at gmail.com> wrote:

> For my purpose, the cachestore delegator is enough...
>
> Well, I'll take a second look at VersionAwareMarshaller...
>
> phil
>
>
> On Thu, Feb 25, 2010 at 3:10 PM, Manik Surtani <manik at jboss.org> wrote:
>
>> I think it makes more sense in the marshaller, simply because the
>> compressed stream is useful elsewhere as well - such as the network, etc.
>>
>> On 25 Feb 2010, at 13:40, Philippe Van Dyck wrote:
>>
>> Sounds like a better idea, closer to the real purpose since the Marshaller
>> is used in other places... but how do I configure this ?
>>
>> Anyway, the marshaller is done, I am testing it, but the code is very easy
>> to adapt to a cachestore...
>>
>> phil
>>
>>
>> On Thu, Feb 25, 2010 at 2:33 PM, Mircea Markus <mircea.markus at jboss.com>wrote:
>>
>>> Can't we handle this through a delegating CacheStore, that would compress
>>> the data and pass it on to the actual store for further processing (the
>>> actual store will take then the binary data and marshall it (ni marshalling
>>> I assume as it receives byte[]).?
>>>
>>> On 25 Feb 2010, at 14:06, Philippe Van Dyck wrote:
>>>
>>> Done
>>>
>>> https://jira.jboss.org/jira/browse/ISPN-357
>>>
>>> Phil
>>>
>>> On Thu, Feb 25, 2010 at 12:22 PM, Manik Surtani <manik at jboss.org> wrote:
>>>
>>>>
>>>> On 25 Feb 2010, at 11:17, Philippe Van Dyck wrote:
>>>>
>>>> Thanks Manik but it is actually related to the compression of the stored
>>>> and (https) transferred ones on the S3 side.
>>>> The S3 bill is 95% lower here... and https transfer time is lower too
>>>> (you pay for booth).
>>>>
>>>>
>>>> Makes sense.  Create a JIRA for this, I'm sure such a feature would be
>>>> generically useful to others as well - if you're happy to contribute your
>>>> work back in.  :)
>>>>
>>>> The Marshaller interface is exposing a lot of methods and it looks like
>>>> the call each other...
>>>>
>>>>
>>>> The original Marshaller interface is legacy, from JBoss Cache, which was
>>>> built as an extension to JGroups' Marshaller interface.  It may look a
>>>> little confusing as a result, but have a look at the AbstractMarshaller
>>>> which should help simplify things.  Further, look at the
>>>> VersionAwareMarshaller, which is the entry point into the marshalling
>>>> system.
>>>>
>>>> HTH
>>>> Manik
>>>>
>>>>
>>>>
>>>>
>>>> phil
>>>>
>>>> On Thu, Feb 25, 2010 at 12:11 PM, Manik Surtani <manik at jboss.org>wrote:
>>>>
>>>>> If it is compression for transmission you are concerned about, you can
>>>>> do this by adding the JGroups COMPRESS protocol to your JGroups cfg.
>>>>>
>>>>> On 25 Feb 2010, at 11:02, philippe van dyck wrote:
>>>>>
>>>>> > Hi All,
>>>>> >
>>>>> > Currently, I compress all data before sending it to the cache. Once
>>>>> compressed, I gain 95% of the JSonized qi4j objects.
>>>>> >
>>>>> > I did some profiling during the load tests and compression is taking
>>>>> roughly 80% of the cpu time.
>>>>> > So I would like to compress only the data sent to the store, not in
>>>>> memory.
>>>>> >
>>>>> > Looks like the Marshaller is my friend here, and I plan to write a
>>>>> compressing wrapper around it.
>>>>> >
>>>>> > Now, when I look at it, I see two ways to wrap the compression
>>>>> process.
>>>>> >
>>>>> > One way is with the ObjectInput / ObjectOutput but I am bothered by
>>>>> the reentrant flag.
>>>>> > The other is the ByteBuffer stuff, no concurrency problem here, but
>>>>> it looks like more work.
>>>>> >
>>>>> > WDYT ?
>>>>> >
>>>>> > Cheers,
>>>>> >
>>>>> > phil
>>>>> > _______________________________________________
>>>>> > infinispan-dev mailing list
>>>>> > infinispan-dev at lists.jboss.org
>>>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>> --
>>>>> Manik Surtani
>>>>> manik at jboss.org
>>>>> Lead, Infinispan
>>>>> Lead, JBoss Cache
>>>>> http://www.infinispan.org
>>>>> http://www.jbosscache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>>>  --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>  --
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100225/6a56c8d1/attachment-0002.html 