On 25 Feb 2010, at 16:49, Manik Surtani wrote:
RE: JIRA, why don't you change it then to say something like "Add
compression to CloudCacheStore"
done
The patch looks good, although I'd recommend using 3 spaces instead of
tabs. ;)
Apart from that, feel free to commit this to trunk yourself, so that your
name is attached to the commit.
Cheers
Manik
Cheers
Manik
On 25 Feb 2010, at 16:14, Philippe Van Dyck wrote:
Ok, tested and working. Here is the patch. Could be improved slightly by
adding the compression level.
Cherry on top, I suffix ".gz" to S3 names, so you can download and
automagically uncompress them ;-)
Manik, what do we do with the JIRA issue ? The patch attached is a bit
unrelated...
phil
On Thu, Feb 25, 2010 at 3:33 PM, Philippe Van Dyck <pvdyck(a)gmail.com>wrote:
> Ok, I need a solution before the end of the day so I will choose the
> easiest way.
>
> I plan to update CloudCacheStoreConfig and add a "compress" option.
> Then I only need to change two simple methods "writeToBlob" and
> "readFromBlob" (thanks Adrian... next time make them protected ;-))
>
> I will post a patch later.
>
> phil
>
>
> On Thu, Feb 25, 2010 at 3:21 PM, Manik Surtani <manik(a)jboss.org> wrote:
>
>>
>> On 25 Feb 2010, at 14:17, Philippe Van Dyck wrote:
>>
>> Another solution is to include a "compression flag" in the version
>> itself.
>>
>>
>> That would work as well... then you could just handle the compression in
>> the VAM
>>
>>
>> WDYT ?
>>
>> phil
>>
>> On Thu, Feb 25, 2010 at 3:15 PM, Philippe Van Dyck
<pvdyck(a)gmail.com>wrote:
>>
>>> For my purpose, the cachestore delegator is enough...
>>>
>>> Well, I'll take a second look at VersionAwareMarshaller...
>>>
>>> phil
>>>
>>>
>>> On Thu, Feb 25, 2010 at 3:10 PM, Manik Surtani <manik(a)jboss.org>
wrote:
>>>
>>>> I think it makes more sense in the marshaller, simply because the
>>>> compressed stream is useful elsewhere as well - such as the network,
etc.
>>>>
>>>> On 25 Feb 2010, at 13:40, Philippe Van Dyck wrote:
>>>>
>>>> Sounds like a better idea, closer to the real purpose since the
>>>> Marshaller is used in other places... but how do I configure this ?
>>>>
>>>> Anyway, the marshaller is done, I am testing it, but the code is very
>>>> easy to adapt to a cachestore...
>>>>
>>>> phil
>>>>
>>>>
>>>> On Thu, Feb 25, 2010 at 2:33 PM, Mircea Markus <
>>>> mircea.markus(a)jboss.com> wrote:
>>>>
>>>>> Can't we handle this through a delegating CacheStore, that would
>>>>> compress the data and pass it on to the actual store for further
processing
>>>>> (the actual store will take then the binary data and marshall it (ni
>>>>> marshalling I assume as it receives byte[]).?
>>>>>
>>>>> On 25 Feb 2010, at 14:06, Philippe Van Dyck wrote:
>>>>>
>>>>> Done
>>>>>
>>>>>
https://jira.jboss.org/jira/browse/ISPN-357
>>>>>
>>>>> Phil
>>>>>
>>>>> On Thu, Feb 25, 2010 at 12:22 PM, Manik Surtani
<manik(a)jboss.org>wrote:
>>>>>
>>>>>>
>>>>>> On 25 Feb 2010, at 11:17, Philippe Van Dyck wrote:
>>>>>>
>>>>>> Thanks Manik but it is actually related to the compression of
the
>>>>>> stored and (https) transferred ones on the S3 side.
>>>>>> The S3 bill is 95% lower here... and https transfer time is lower
too
>>>>>> (you pay for booth).
>>>>>>
>>>>>>
>>>>>> Makes sense. Create a JIRA for this, I'm sure such a feature
would
>>>>>> be generically useful to others as well - if you're happy to
contribute your
>>>>>> work back in. :)
>>>>>>
>>>>>> The Marshaller interface is exposing a lot of methods and it
looks
>>>>>> like the call each other...
>>>>>>
>>>>>>
>>>>>> The original Marshaller interface is legacy, from JBoss Cache,
which
>>>>>> was built as an extension to JGroups' Marshaller interface.
It may look a
>>>>>> little confusing as a result, but have a look at the
AbstractMarshaller
>>>>>> which should help simplify things. Further, look at the
>>>>>> VersionAwareMarshaller, which is the entry point into the
marshalling
>>>>>> system.
>>>>>>
>>>>>> HTH
>>>>>> Manik
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> phil
>>>>>>
>>>>>> On Thu, Feb 25, 2010 at 12:11 PM, Manik Surtani
<manik(a)jboss.org>wrote:
>>>>>>
>>>>>>> If it is compression for transmission you are concerned
about, you
>>>>>>> can do this by adding the JGroups COMPRESS protocol to your
JGroups cfg.
>>>>>>>
>>>>>>> On 25 Feb 2010, at 11:02, philippe van dyck wrote:
>>>>>>>
>>>>>>> > Hi All,
>>>>>>> >
>>>>>>> > Currently, I compress all data before sending it to the
cache.
>>>>>>> Once compressed, I gain 95% of the JSonized qi4j objects.
>>>>>>> >
>>>>>>> > I did some profiling during the load tests and
compression is
>>>>>>> taking roughly 80% of the cpu time.
>>>>>>> > So I would like to compress only the data sent to the
store, not
>>>>>>> in memory.
>>>>>>> >
>>>>>>> > Looks like the Marshaller is my friend here, and I plan
to write a
>>>>>>> compressing wrapper around it.
>>>>>>> >
>>>>>>> > Now, when I look at it, I see two ways to wrap the
compression
>>>>>>> process.
>>>>>>> >
>>>>>>> > One way is with the ObjectInput / ObjectOutput but I am
bothered
>>>>>>> by the reentrant flag.
>>>>>>> > The other is the ByteBuffer stuff, no concurrency
problem here,
>>>>>>> but it looks like more work.
>>>>>>> >
>>>>>>> > WDYT ?
>>>>>>> >
>>>>>>> > Cheers,
>>>>>>> >
>>>>>>> > phil
>>>>>>> > _______________________________________________
>>>>>>> > infinispan-dev mailing list
>>>>>>> > infinispan-dev(a)lists.jboss.org
>>>>>>> >
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>
>>>>>>> --
>>>>>>> Manik Surtani
>>>>>>> manik(a)jboss.org
>>>>>>> Lead, Infinispan
>>>>>>> Lead, JBoss Cache
>>>>>>>
http://www.infinispan.org
>>>>>>>
http://www.jbosscache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Manik Surtani
>>>>>> manik(a)jboss.org
>>>>>> Lead, Infinispan
>>>>>> Lead, JBoss Cache
>>>>>>
http://www.infinispan.org
>>>>>>
http://www.jbosscache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev(a)lists.jboss.org
>>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev(a)lists.jboss.org
>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev(a)lists.jboss.org
>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev(a)lists.jboss.org
>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>>> --
>>>> Manik Surtani
>>>> manik(a)jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>>
http://www.infinispan.org
>>>>
http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev(a)lists.jboss.org
>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Manik Surtani
>> manik(a)jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>>
http://www.infinispan.org
>>
http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
<cloud.patch>_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev