Hi Adrian, I will be doing some heavy load testing in the next weeks, so I'll be glad to test it, with Infinispan ! So the question is, will you update the Infinispan client too ? cheers, phil Le 5 févr. 2010 à 07:01, Adrian Cole a écrit : > Many of you know that the java and nio http engines and jclouds have > their own respective warts: > > * the nio one is unreliable, as connection breaks under load are not > self-healing. > * the default java one uses too many connections and memory under load. > > Sam Tunnicliffe sent us a patch a while back to add support for apache > hc 4.0 [1]. I've revised this and committed to trunk/1.0-SNAPSHOT. > I've also included this in the enterprise module. Here's how to use > that module: > > http://code.google.com/p/jclouds/wiki/jcloudsAPI > > Please help kick the tires on this, as it is an important piece of our > core infrastructure. If you see any opportunities for improvement, do > shout out. > > Thanks, > -Adrian > > [1] http://code.google.com/p/jclouds/issues/detail?id=107 > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Are you considering to include this in the next Infinispan release ? Cheers, Sanne 2010/2/5 Manik Surtani <manik(a)jboss.org&gt;: > Adrian, once you have published the snapshot to your repo, we'd need to: > > 1) Update CloudCacheStore to use this snapshot > 2) Add provisions to be able to tune the CCS with maxConnectionsPerContext, maxConnectionsPerHost, ioThreads and userThreads. Actually, could mCPContext and mCPHost be combined into a single maxConnections setting for the CCS, since we only create 1 context per instance? > 3) Test, and release JClouds 1.0-beta-4 or cr-1. I'd like to keep dependency on a snapshot to a very minimum timeframe. > > Cheers > Manik > > On 5 Feb 2010, at 07:37, philippe van dyck wrote: > >> Hi Adrian, >> >> I will be doing some heavy load testing in the next weeks, so I'll be glad to test it, with Infinispan ! >> So the question is, will you update the Infinispan client too ? >> >> cheers, >> >> phil >> >> >> Le 5 févr. 2010 à 07:01, Adrian Cole a écrit : >> >>> Many of you know that the java and nio http engines and jclouds have >>> their own respective warts: >>> >>> * the nio one is unreliable, as connection breaks under load are not >>> self-healing. >>> * the default java one uses too many connections and memory under load. >>> >>> Sam Tunnicliffe sent us a patch a while back to add support for apache >>> hc 4.0 [1]. I've revised this and committed to trunk/1.0-SNAPSHOT. >>> I've also included this in the enterprise module. Here's how to use >>> that module: >>> >>> http://code.google.com/p/jclouds/wiki/jcloudsAPI >>> >>> Please help kick the tires on this, as it is an important piece of our >>> core infrastructure. If you see any opportunities for improvement, do >>> shout out. >>> >>> Thanks, >>> -Adrian >>> >>> [1] http://code.google.com/p/jclouds/issues/detail?id=107 >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Hi, guys. I think we are ok releasing a beta-4 that includes this fix and other glitches we've found after this is shaken out.  That can happen in a couple days. maven SNAPSHOT job hasn't been moved to git, yet.  Until we let you know, testing this patch would involve a local build.  Should be ok by tomorrow. I hope this helps, -Adrian On Fri, Feb 5, 2010 at 5:40 AM, Manik Surtani <manik(a)jboss.org&gt; wrote: > Depends on when it comes in.  If we are talking a few days, then yes.  Otherwise, either a point release of 4.0 or 4.1. > > On 5 Feb 2010, at 12:15, Sanne Grinovero wrote: > >> Are you considering to include this in the next Infinispan release ? >> >> Cheers, >> Sanne >> >> 2010/2/5 Manik Surtani <manik(a)jboss.org&gt;: >>> Adrian, once you have published the snapshot to your repo, we'd need to: >>> >>> 1) Update CloudCacheStore to use this snapshot >>> 2) Add provisions to be able to tune the CCS with maxConnectionsPerContext, maxConnectionsPerHost, ioThreads and userThreads.  Actually, could mCPContext and mCPHost be combined into a single maxConnections setting for the CCS, since we only create 1 context per instance? >>> 3) Test, and release JClouds 1.0-beta-4 or cr-1.  I'd like to keep dependency on a snapshot to a very minimum timeframe. >>> >>> Cheers >>> Manik >>> >>> On 5 Feb 2010, at 07:37, philippe van dyck wrote: >>> >>>> Hi Adrian, >>>> >>>> I will be doing some heavy load testing in the next weeks, so I'll be glad to test it, with Infinispan ! >>>> So the question is, will you update the Infinispan client too ? >>>> >>>> cheers, >>>> >>>> phil >>>> >>>> >>>> Le 5 févr. 2010 à 07:01, Adrian Cole a écrit : >>>> >>>>> Many of you know that the java and nio http engines and jclouds have >>>>> their own respective warts: >>>>> >>>>> * the nio one is unreliable, as connection breaks under load are not >>>>> self-healing. >>>>> * the default java one uses too many connections and memory under load. >>>>> >>>>> Sam Tunnicliffe sent us a patch a while back to add support for apache >>>>> hc 4.0 [1].  I've revised this and committed to trunk/1.0-SNAPSHOT. >>>>> I've also included this in the enterprise module.  Here's how to use >>>>> that module: >>>>> >>>>> http://code.google.com/p/jclouds/wiki/jcloudsAPI >>>>> >>>>> Please help kick the tires on this, as it is an important piece of our >>>>> core infrastructure.  If you see any opportunities for improvement, do >>>>> shout out. >>>>> >>>>> Thanks, >>>>> -Adrian >>>>> >>>>> [1] http://code.google.com/p/jclouds/issues/detail?id=107 >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> -- >>> Manik Surtani >>> manik(a)jboss.org >>> Lead, Infinispan >>> Lead, JBoss Cache >>> http://www.infinispan.org >>> http://www.jbosscache.org >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev >

If you don't mind, since there is "no" limit from ec2 to s3 (http://developer.amazonwebservices.com/connect/thread.jspa?messageID=1533...), I would like to improve reliability first. The actual fine tuning will never be achieved, since it depends on the size of the cache entries, unless we find an http client with throughput management (QoS). An ac2 small instance is supposed to deliver 20MBPS ... but frankly, I have doubts. So I would like the CacheStore config to expose a maximum of options, since it the actual optimization will probably happen very late in the development process. cheers, phil Le 6 févr. 2010 à 08:54, Adrian Cole a écrit : > The snapshot should be up to date, now. > > The challenge is to find the best tuning options out of the below: > > http://code.google.com/p/jclouds/wiki/jcloudsAPI#Enterprise > > Let the games begin ;) > > -Adrian > > On Fri, Feb 5, 2010 at 8:05 AM, Adrian Cole <adrian.f.cole(a)gmail.com&gt; wrote: >> Hi, guys. >> >> I think we are ok releasing a beta-4 that includes this fix and other >> glitches we've found after this is shaken out.  That can happen in a >> couple days. >> >> maven SNAPSHOT job hasn't been moved to git, yet.  Until we let you >> know, testing this patch would involve a local build.  Should be ok by >> tomorrow. >> >> I hope this helps, >> -Adrian >> >> On Fri, Feb 5, 2010 at 5:40 AM, Manik Surtani <manik(a)jboss.org&gt; wrote: >>> Depends on when it comes in.  If we are talking a few days, then yes.  Otherwise, either a point release of 4.0 or 4.1. >>> >>> On 5 Feb 2010, at 12:15, Sanne Grinovero wrote: >>> >>>> Are you considering to include this in the next Infinispan release ? >>>> >>>> Cheers, >>>> Sanne >>>> >>>> 2010/2/5 Manik Surtani <manik(a)jboss.org&gt;: >>>>> Adrian, once you have published the snapshot to your repo, we'd need to: >>>>> >>>>> 1) Update CloudCacheStore to use this snapshot >>>>> 2) Add provisions to be able to tune the CCS with maxConnectionsPerContext, maxConnectionsPerHost, ioThreads and userThreads.  Actually, could mCPContext and mCPHost be combined into a single maxConnections setting for the CCS, since we only create 1 context per instance? >>>>> 3) Test, and release JClouds 1.0-beta-4 or cr-1.  I'd like to keep dependency on a snapshot to a very minimum timeframe. >>>>> >>>>> Cheers >>>>> Manik >>>>> >>>>> On 5 Feb 2010, at 07:37, philippe van dyck wrote: >>>>> >>>>>> Hi Adrian, >>>>>> >>>>>> I will be doing some heavy load testing in the next weeks, so I'll be glad to test it, with Infinispan ! >>>>>> So the question is, will you update the Infinispan client too ? >>>>>> >>>>>> cheers, >>>>>> >>>>>> phil >>>>>> >>>>>> >>>>>> Le 5 févr. 2010 à 07:01, Adrian Cole a écrit : >>>>>> >>>>>>> Many of you know that the java and nio http engines and jclouds have >>>>>>> their own respective warts: >>>>>>> >>>>>>> * the nio one is unreliable, as connection breaks under load are not >>>>>>> self-healing. >>>>>>> * the default java one uses too many connections and memory under load. >>>>>>> >>>>>>> Sam Tunnicliffe sent us a patch a while back to add support for apache >>>>>>> hc 4.0 [1].  I've revised this and committed to trunk/1.0-SNAPSHOT. >>>>>>> I've also included this in the enterprise module.  Here's how to use >>>>>>> that module: >>>>>>> >>>>>>> http://code.google.com/p/jclouds/wiki/jcloudsAPI >>>>>>> >>>>>>> Please help kick the tires on this, as it is an important piece of our >>>>>>> core infrastructure.  If you see any opportunities for improvement, do >>>>>>> shout out. >>>>>>> >>>>>>> Thanks, >>>>>>> -Adrian >>>>>>> >>>>>>> [1] http://code.google.com/p/jclouds/issues/detail?id=107 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> infinispan-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>>> -- >>>>> Manik Surtani >>>>> manik(a)jboss.org >>>>> Lead, Infinispan >>>>> Lead, JBoss Cache >>>>> http://www.infinispan.org >>>>> http://www.jbosscache.org >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> -- >>> Manik Surtani >>> manik(a)jboss.org >>> Lead, Infinispan >>> Lead, JBoss Cache >>> http://www.infinispan.org >>> http://www.jbosscache.org >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 6 Feb 2010, at 11:55, philippe van dyck wrote: > If you don't mind, since there is "no" limit from ec2 to s3 (http://developer.amazonwebservices.com/connect/thread.jspa?messageID=1533...), I would like to improve reliability first. > The actual fine tuning will never be achieved, since it depends on the size of the cache entries, unless we find an http client with throughput management (QoS). > > An ac2 small instance is supposed to deliver 20MBPS ... but frankly, I have doubts. > So I would like the CacheStore config to expose a maximum of options, since it the actual optimization will probably happen very late in the development process. I agree - we should expose all tuning options as cachestore config options. Sensible defaults though, as always.

> > cheers, > > phil > > Le 6 févr. 2010 à 08:54, Adrian Cole a écrit : > >> The snapshot should be up to date, now. >> >> The challenge is to find the best tuning options out of the below: >> >> http://code.google.com/p/jclouds/wiki/jcloudsAPI#Enterprise >> >> Let the games begin ;) >> >> -Adrian >> >> On Fri, Feb 5, 2010 at 8:05 AM, Adrian Cole <adrian.f.cole(a)gmail.com&gt; wrote: >>> Hi, guys. >>> >>> I think we are ok releasing a beta-4 that includes this fix and other >>> glitches we've found after this is shaken out. That can happen in a >>> couple days. >>> >>> maven SNAPSHOT job hasn't been moved to git, yet. Until we let you >>> know, testing this patch would involve a local build. Should be ok by >>> tomorrow. >>> >>> I hope this helps, >>> -Adrian >>> >>> On Fri, Feb 5, 2010 at 5:40 AM, Manik Surtani <manik(a)jboss.org&gt; wrote: >>>> Depends on when it comes in. If we are talking a few days, then yes. Otherwise, either a point release of 4.0 or 4.1. >>>> >>>> On 5 Feb 2010, at 12:15, Sanne Grinovero wrote: >>>> >>>>> Are you considering to include this in the next Infinispan release ? >>>>> >>>>> Cheers, >>>>> Sanne >>>>> >>>>> 2010/2/5 Manik Surtani <manik(a)jboss.org&gt;: >>>>>> Adrian, once you have published the snapshot to your repo, we'd need to: >>>>>> >>>>>> 1) Update CloudCacheStore to use this snapshot >>>>>> 2) Add provisions to be able to tune the CCS with maxConnectionsPerContext, maxConnectionsPerHost, ioThreads and userThreads. Actually, could mCPContext and mCPHost be combined into a single maxConnections setting for the CCS, since we only create 1 context per instance? >>>>>> 3) Test, and release JClouds 1.0-beta-4 or cr-1. I'd like to keep dependency on a snapshot to a very minimum timeframe. >>>>>> >>>>>> Cheers >>>>>> Manik >>>>>> >>>>>> On 5 Feb 2010, at 07:37, philippe van dyck wrote: >>>>>> >>>>>>> Hi Adrian, >>>>>>> >>>>>>> I will be doing some heavy load testing in the next weeks, so I'll be glad to test it, with Infinispan ! >>>>>>> So the question is, will you update the Infinispan client too ? >>>>>>> >>>>>>> cheers, >>>>>>> >>>>>>> phil >>>>>>> >>>>>>> >>>>>>> Le 5 févr. 2010 à 07:01, Adrian Cole a écrit : >>>>>>> >>>>>>>> Many of you know that the java and nio http engines and jclouds have >>>>>>>> their own respective warts: >>>>>>>> >>>>>>>> * the nio one is unreliable, as connection breaks under load are not >>>>>>>> self-healing. >>>>>>>> * the default java one uses too many connections and memory under load. >>>>>>>> >>>>>>>> Sam Tunnicliffe sent us a patch a while back to add support for apache >>>>>>>> hc 4.0 [1]. I've revised this and committed to trunk/1.0-SNAPSHOT. >>>>>>>> I've also included this in the enterprise module. Here's how to use >>>>>>>> that module: >>>>>>>> >>>>>>>> http://code.google.com/p/jclouds/wiki/jcloudsAPI >>>>>>>> >>>>>>>> Please help kick the tires on this, as it is an important piece of our >>>>>>>> core infrastructure. If you see any opportunities for improvement, do >>>>>>>> shout out. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -Adrian >>>>>>>> >>>>>>>> [1] http://code.google.com/p/jclouds/issues/detail?id=107 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> infinispan-dev mailing list >>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> infinispan-dev mailing list >>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>>> -- >>>>>> Manik Surtani >>>>>> manik(a)jboss.org >>>>>> Lead, Infinispan >>>>>> Lead, JBoss Cache >>>>>> http://www.infinispan.org >>>>>> http://www.jbosscache.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> infinispan-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> -- >>>> Manik Surtani >>>> manik(a)jboss.org >>>> Lead, Infinispan >>>> Lead, JBoss Cache >>>> http://www.infinispan.org >>>> http://www.jbosscache.org >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev