Re: [infinispan-dev] Infinispan S3 Cluster + Store Success ! (+serious performance problems)

Thanks for your help Bela. Indeed, when I replace the S3 cache store with the disk one, the performance problem disappears (takes less than a second to store my 100 'put' updates when I commit the transaction). Here is my config : <?xml version="1.0" encoding="UTF-8"?> <infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:infinispan:config:4.0"> <global> <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport"> <properties> <property name="configurationFile" value="jgroups.xml" /> </properties> </transport> </global> <default> <transaction transactionManagerLookupClass="org.infinispan.transaction.lookup.DummyTransactionManagerLookup" /> <clustering mode="distribution"> <l1 enabled="true" lifespan="100000" /> <hash numOwners="2" rehashRpcTimeout="120000" /> </clustering> <loaders passivation="false" shared="true" preload="false"> <loader class="org.infinispan.loaders.s3.S3CacheStore" fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false"> <properties> <property name="awsAccessKey" value="xxx" /> <property name="awsSecretKey" value="xxx" /> <property name="bucketPrefix" value="store" /> </properties> <async enabled="true"/> </loader> </loaders> <unsafe unreliableReturnValues="true" /> </default> </infinispan> And the log : INFO (14:54:37): JGroupsTransport - Starting JGroups Channel INFO (14:54:38): JChannel - JGroups version: 2.8.0.CR5 ------------------------------------------------------------------- GMS: address=sakapuss.local-16157, cluster=Infinispan-Cluster, physical address=192.168.1.136:7800 ------------------------------------------------------------------- INFO (14:54:49): JGroupsTransport - Received new cluster view: [sakapuss.local-16157|0] [sakapuss.local-16157] INFO (14:54:49): JGroupsTransport - Cache local address is sakapuss.local-16157, physical address is 192.168.1.136:7800 INFO (14:54:49): GlobalComponentRegistry - Infinispan version: Infinispan 'Starobrno' 4.0.0.CR2 INFO (14:54:49): AsyncStore - Async cache loader starting org.infinispan.loaders.decorators.AsyncStore@7254d7ac WARN (14:54:51): utureCommandConnectionPool - org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnectionPool@fcd4eca1 - saturated connection pool INFO (14:54:52): ComponentRegistry - Infinispan version: Infinispan 'Starobrno' 4.0.0.CR2 please note the HttpNioFutureCommandConnectionPool@fcd4eca1 - saturated connection pool (??) Philippe Le 2 déc. 2009 à 14:28, Bela Ban a écrit : > Just to narrow down the issue: when you disable the S3 cache store, I > assume the performance problem goes away, correct ? > > Just trying to pin the blame on the S3 cache loader, then I don't even > need to see whether it is a JGroups problem... :-) > > > > philippe van dyck wrote: > >> Hi Infinispan mailing list, >> >> a couple of days ago, I succeeded in writing an entity store for qi4j (http://www.qi4j.org/) using Infinispan, the S3 store and the S3_PING JGroups clustering configuration. >> >> It works like a charm, discovers new EC2 instances, synchronizes and process transactions perfectly... you did an amazing job. >> >> But I have a serious performance problems. >> >> When I write an update (<1k) to the cache, it takes around 500 ms to be stored on S3. >> >> The best result I achieved was around 10 cache writes per second... it is abysmal (when using httpclient directly I had a min of 100/sec using 20 connections). >> >> When I commit a JTA transaction made of 100 cache writes, it takes around 30 seconds (cpu<5%) and the first write ends on S3 after at least 5 seconds of 'idle' time (SSL negotiation??). >> >> I disabled the store asynchronism and work without JTA transactions, no effect on performance. >> >> I also modified the jClouds configuration, multiplying by 10 all worker threads, connections and the rest... no improvement! >> >> When I (load) test my web app (wicket based+qi4j+...infinispan) the cpu stays idle (<5%) and ... JTA transactions fails (timeouts) and I cannot acquire locks before the 10 seconds timeout. >> >> Is there something fishy in the jclouds configuration ? in the httpnio use of jclouds ? in the version of jclouds (the trunk one with the blob store seems to be so different) ? >> >> Am I missing something ? >> >> Any pointer to any doc/help/experience is welcome ;-) >> >> Philippe >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> > -- > Bela Ban > Lead JGroups / Clustering Team > JBoss > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > ------------------------------------------------------------------------ _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev