Re: [infinispan-dev] Compressing Marshaller Wrapper

On 25 Feb 2010, at 14:17, Philippe Van Dyck wrote: Another solution is to include a "compression flag" in the version itself. That would work as well... then you could just handle the compression in the VAM WDYT ? phil On Thu, Feb 25, 2010 at 3:15 PM, Philippe Van Dyck <pvdyck(a)gmail.com&gt;wrote: > For my purpose, the cachestore delegator is enough... > > Well, I'll take a second look at VersionAwareMarshaller... > > phil > > > On Thu, Feb 25, 2010 at 3:10 PM, Manik Surtani <manik(a)jboss.org&gt; wrote: > >> I think it makes more sense in the marshaller, simply because the >> compressed stream is useful elsewhere as well - such as the network, etc. >> >> On 25 Feb 2010, at 13:40, Philippe Van Dyck wrote: >> >> Sounds like a better idea, closer to the real purpose since the >> Marshaller is used in other places... but how do I configure this ? >> >> Anyway, the marshaller is done, I am testing it, but the code is very >> easy to adapt to a cachestore... >> >> phil >> >> >> On Thu, Feb 25, 2010 at 2:33 PM, Mircea Markus <mircea.markus(a)jboss.com&gt;wrote: >> >>> Can't we handle this through a delegating CacheStore, that would >>> compress the data and pass it on to the actual store for further processing >>> (the actual store will take then the binary data and marshall it (ni >>> marshalling I assume as it receives byte[]).? >>> >>> On 25 Feb 2010, at 14:06, Philippe Van Dyck wrote: >>> >>> Done >>> >>> https://jira.jboss.org/jira/browse/ISPN-357 >>> >>> Phil >>> >>> On Thu, Feb 25, 2010 at 12:22 PM, Manik Surtani <manik(a)jboss.org&gt;wrote: >>> >>>> >>>> On 25 Feb 2010, at 11:17, Philippe Van Dyck wrote: >>>> >>>> Thanks Manik but it is actually related to the compression of the >>>> stored and (https) transferred ones on the S3 side. >>>> The S3 bill is 95% lower here... and https transfer time is lower too >>>> (you pay for booth). >>>> >>>> >>>> Makes sense. Create a JIRA for this, I'm sure such a feature would be >>>> generically useful to others as well - if you're happy to contribute your >>>> work back in. :) >>>> >>>> The Marshaller interface is exposing a lot of methods and it looks like >>>> the call each other... >>>> >>>> >>>> The original Marshaller interface is legacy, from JBoss Cache, which >>>> was built as an extension to JGroups' Marshaller interface. It may look a >>>> little confusing as a result, but have a look at the AbstractMarshaller >>>> which should help simplify things. Further, look at the >>>> VersionAwareMarshaller, which is the entry point into the marshalling >>>> system. >>>> >>>> HTH >>>> Manik >>>> >>>> >>>> >>>> >>>> phil >>>> >>>> On Thu, Feb 25, 2010 at 12:11 PM, Manik Surtani <manik(a)jboss.org&gt;wrote: >>>> >>>>> If it is compression for transmission you are concerned about, you can >>>>> do this by adding the JGroups COMPRESS protocol to your JGroups cfg. >>>>> >>>>> On 25 Feb 2010, at 11:02, philippe van dyck wrote: >>>>> >>>>> > Hi All, >>>>> > >>>>> > Currently, I compress all data before sending it to the cache. Once >>>>> compressed, I gain 95% of the JSonized qi4j objects. >>>>> > >>>>> > I did some profiling during the load tests and compression is taking >>>>> roughly 80% of the cpu time. >>>>> > So I would like to compress only the data sent to the store, not in >>>>> memory. >>>>> > >>>>> > Looks like the Marshaller is my friend here, and I plan to write a >>>>> compressing wrapper around it. >>>>> > >>>>> > Now, when I look at it, I see two ways to wrap the compression >>>>> process. >>>>> > >>>>> > One way is with the ObjectInput / ObjectOutput but I am bothered by >>>>> the reentrant flag. >>>>> > The other is the ByteBuffer stuff, no concurrency problem here, but >>>>> it looks like more work. >>>>> > >>>>> > WDYT ? >>>>> > >>>>> > Cheers, >>>>> > >>>>> > phil >>>>> > _______________________________________________ >>>>> > infinispan-dev mailing list >>>>> > infinispan-dev(a)lists.jboss.org >>>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>>> -- >>>>> Manik Surtani >>>>> manik(a)jboss.org >>>>> Lead, Infinispan >>>>> Lead, JBoss Cache >>>>> http://www.infinispan.org >>>>> http://www.jbosscache.org >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> -- >>>> Manik Surtani >>>> manik(a)jboss.org >>>> Lead, Infinispan >>>> Lead, JBoss Cache >>>> http://www.infinispan.org >>>> http://www.jbosscache.org >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev