[infinispan-dev] please help test new apache hc 4.0 support in jclouds

[Adrian Cole]

I don't mind ;)  I have heard that the maximum bandwith to S3 from an
account is 12MBPs (at an ec2 enterprise meeting a few months ago).  I
haven't tried that.  The "speed test" [1] I ran in the past was able
to run out of file descriptors sending to SQS, as at some point the
TPS were as much as 1600.  I haven't re-run this using apache HC,
though.

Anyway, the settings I mentioned can be set to zero which will make
them unlimited.

I hope this helps.
-Adrian
[1]http://anyweight.blogspot.com/2010/01/run-your-own-sqs-speed-test.html

On Sat, Feb 6, 2010 at 3:55 AM, philippe van dyck <pvdyck at gmail.com> wrote:
> If you don't mind, since there is "no" limit from ec2 to s3 (http://developer.amazonwebservices.com/connect/thread.jspa?messageID=153314&#153314), I would like to improve reliability first.
> The actual fine tuning will never be achieved, since it depends on the size of the cache entries, unless we find an http client with throughput management (QoS).
>
> An ac2 small instance is supposed to deliver 20MBPS ... but frankly, I have doubts.
> So I would like the CacheStore config to expose a maximum of options, since it the actual optimization will probably happen very late in the development process.
>
> cheers,
>
> phil
>
> Le 6 févr. 2010 à 08:54, Adrian Cole a écrit :
>
>> The snapshot should be up to date, now.
>>
>> The challenge is to find the best tuning options out of the below:
>>
>> http://code.google.com/p/jclouds/wiki/jcloudsAPI#Enterprise
>>
>> Let the games begin ;)
>>
>> -Adrian
>>
>> On Fri, Feb 5, 2010 at 8:05 AM, Adrian Cole <adrian.f.cole at gmail.com> wrote:
>>> Hi, guys.
>>>
>>> I think we are ok releasing a beta-4 that includes this fix and other
>>> glitches we've found after this is shaken out.  That can happen in a
>>> couple days.
>>>
>>> maven SNAPSHOT job hasn't been moved to git, yet.  Until we let you
>>> know, testing this patch would involve a local build.  Should be ok by
>>> tomorrow.
>>>
>>> I hope this helps,
>>> -Adrian
>>>
>>> On Fri, Feb 5, 2010 at 5:40 AM, Manik Surtani <manik at jboss.org> wrote:
>>>> Depends on when it comes in.  If we are talking a few days, then yes.  Otherwise, either a point release of 4.0 or 4.1.
>>>>
>>>> On 5 Feb 2010, at 12:15, Sanne Grinovero wrote:
>>>>
>>>>> Are you considering to include this in the next Infinispan release ?
>>>>>
>>>>> Cheers,
>>>>> Sanne
>>>>>
>>>>> 2010/2/5 Manik Surtani <manik at jboss.org>:
>>>>>> Adrian, once you have published the snapshot to your repo, we'd need to:
>>>>>>
>>>>>> 1) Update CloudCacheStore to use this snapshot
>>>>>> 2) Add provisions to be able to tune the CCS with maxConnectionsPerContext, maxConnectionsPerHost, ioThreads and userThreads.  Actually, could mCPContext and mCPHost be combined into a single maxConnections setting for the CCS, since we only create 1 context per instance?
>>>>>> 3) Test, and release JClouds 1.0-beta-4 or cr-1.  I'd like to keep dependency on a snapshot to a very minimum timeframe.
>>>>>>
>>>>>> Cheers
>>>>>> Manik
>>>>>>
>>>>>> On 5 Feb 2010, at 07:37, philippe van dyck wrote:
>>>>>>
>>>>>>> Hi Adrian,
>>>>>>>
>>>>>>> I will be doing some heavy load testing in the next weeks, so I'll be glad to test it, with Infinispan !
>>>>>>> So the question is, will you update the Infinispan client too ?
>>>>>>>
>>>>>>> cheers,
>>>>>>>
>>>>>>> phil
>>>>>>>
>>>>>>>
>>>>>>> Le 5 févr. 2010 à 07:01, Adrian Cole a écrit :
>>>>>>>
>>>>>>>> Many of you know that the java and nio http engines and jclouds have
>>>>>>>> their own respective warts:
>>>>>>>>
>>>>>>>> * the nio one is unreliable, as connection breaks under load are not
>>>>>>>> self-healing.
>>>>>>>> * the default java one uses too many connections and memory under load.
>>>>>>>>
>>>>>>>> Sam Tunnicliffe sent us a patch a while back to add support for apache
>>>>>>>> hc 4.0 [1].  I've revised this and committed to trunk/1.0-SNAPSHOT.
>>>>>>>> I've also included this in the enterprise module.  Here's how to use
>>>>>>>> that module:
>>>>>>>>
>>>>>>>> http://code.google.com/p/jclouds/wiki/jcloudsAPI
>>>>>>>>
>>>>>>>> Please help kick the tires on this, as it is an important piece of our
>>>>>>>> core infrastructure.  If you see any opportunities for improvement, do
>>>>>>>> shout out.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -Adrian
>>>>>>>>
>>>>>>>> [1] http://code.google.com/p/jclouds/issues/detail?id=107
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>> --
>>>>>> Manik Surtani
>>>>>> manik at jboss.org
>>>>>> Lead, Infinispan
>>>>>> Lead, JBoss Cache
>>>>>> http://www.infinispan.org
>>>>>> http://www.jbosscache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>