[infinispan-dev] Infinispan Large Scale support

[david marion]

Bela, Manik,
 
 Thanks for responding. Will ISPN-83 be included in 4.2.1.FINAL? Yes, a large cluster jgroups config would be great. At this point we have taken the UDP config distributed with 4.2.0 and increased all the timeouts. It took about 30 minutes to get to 150 nodes. The cluster appears stable once its up, the problem is in the startup. Based on what I am seeing and have read in the documentation, every time a node wants to join a FLUSH is sent across the entire system and then a new view is created. We are seeing nodes wait minutes to get the new view. It has been mentioned before that Infinispan was tested at 1000 nodes and there may be a couple of places where Infinispan is in use at the several hundred node size. Can we get a copy of the configuration files for these systems?
 
Dave Marion
 


From: manik at jboss.org
Date: Fri, 18 Mar 2011 10:22:05 +0000
To: infinispan-dev at lists.jboss.org
Subject: Re: [infinispan-dev] Infinispan Large Scale support





On 18 Mar 2011, at 07:12, Bela Ban wrote:



Initially we were seeing lots of little clusters form, increasing the number of initial members in the PING section of the jgroups-udp.xml file and increasing the

timeouts seems to have made things better but not great. Even at 20 
nodes, we are seeing messages where the coordinator is failing to flush.

I do *not* recommend FLUSH in clusters bigger than 10-15 nodes ! FLUSH 
will definitely kill you if you have clusters of 100s of nodes.

IIRC, Infinispan requires FLUSH, but I think this was removed in 
4.2.1.FINAL. Maybe Manik or Vladimir can comment ?



4.2.1.FINAL isn't out yet, but it should be soon.


The strategy I'm taking is that FLUSH will still be enabled and required to maintain the same behaviour as 4.2.0, but you will be able to switch this check off with a system property flag, and provide a JGroups config that excludes FLUSH.



It does not appear that the default configuration scales to a large size.


Correct. The default config is geared toward a 4-16 node cluster.



Yes, you will need to tune your JGroups config for larger clusters.  Maybe it makes sense to ship a "jgroups-large-cluster.xml" config as well, to help ease the complexity of this in future.


Cheers
Manik



--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani


Lead, Infinispan
http://www.infinispan.org



_______________________________________________ infinispan-dev mailing list infinispan-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20110318/cf80fd43/attachment.html