[infinispan-dev] Infinispan Large Scale support

[Dave]

  Won't be able to get CR4 uploaded, policy dictates that I wait until final
release. However, I was able to get 431 nodes up and running as a replicated
cluster and 115 nodes up as a distributed cluster. For the 430 node cache, I
was able to get it started with no problems about 50% of the time. When they
formed multiple clusters they merged together only some of the time. It
really does appear to be a startup issue at this point. We have not pushed
it hard enough yet to see what happens at this scale under load. 

 

Any idea when CR4 will be FINAL?

Are there any tools to help diagnose problems / performance at this scale (I
ended up writing my own monitor program)?

 

Dave Marion

 

-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org
[mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of david marion
Sent: Friday, March 18, 2011 12:26 PM
To: infinispan-dev at lists.jboss.org
Subject: Re: [infinispan-dev] Infinispan Large Scale support

 

 
  I will see if I can get 4.2.1.CR4 uploaded onto the system. Is there a
reference to the system property in question?
 

  _____  

From: manik at jboss.org
Date: Fri, 18 Mar 2011 16:07:06 +0000
To: infinispan-dev at lists.jboss.org
Subject: Re: [infinispan-dev] Infinispan Large Scale support



On 18 Mar 2011, at 15:45, david marion wrote:

 

Bela,
 
  Agreed, increasing the timeouts is probably not the way to go. However I
am at the mercy of whatever Infinispan is doing. As for the notion of
Infinispan being tested at 1000 nodes:
 
  Manik was quoted here:
http://itmanagement.earthweb.com/netsys/article.php/3864436/Red-Hat-Ramps-Up
-Open-Source-Cloud-Projects.htm

 

Yeah, we got a hold of an 1100 node cluster to test on at that time.  But
that cluster proved unstable and kept falling over with network issues.

 

  It was never denied here:
http://community.jboss.org/thread/156494?tstart=0
  and I'll have to wait until I get home to dig up an email that I think I
have.
 
However, I'm not assigning blame, I was under the impression that it had
been tested at 1000 nodes. I just had it up at 430 nodes, but I can't
reliably get it to that size when I restart it. I think the answer is that
Infinispan will have to use what will scale in JGroups (remove what does not
scale). Until then I will have to scale it down to a size that I can start
in a reliable fashion. 
 
  Still no answer on whether ISPN-83 will be in 4.2.1.....

 

See my response earlier on this thread about this.

 

Specifically, Infinispan throws a config exception if FLUSH is *not*
present.  However, we did cut a release without this check and ran our
defaults without FLUSH.  It worked well enough for most people, except one
important use case.  And this is what needs further investigation.  It seems
as though the problem with that use case was not the removal of FLUSH, but
some dependence in the test in question on FLUSH.  But it's too late in the
4.2.1 cycle to pull something like this, hence my proposal to leave FLUSH in
by default but to allow a FLUSH-free config via a (temporary, 4.2.x only)
system property while we investigate properly removing FLUSH in 5.0.

 

Cheers

Manik

 

--

Manik Surtani

manik at jboss.org

twitter.com/maniksurtani

 

Lead, Infinispan

http://www.infinispan.org <http://www.infinispan.org/> 

 

 



_______________________________________________ infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20110318/2938da44/attachment-0001.html