[infinispan-dev] Improvements for SessionSimulatorTest in benchmark framework

[Manik Surtani]

On 2 Jun 2009, at 11:40, Galder Zamarreno wrote:

> Hi all,
>
> Over the weekend, I've been reading the JCIP and I've come up with a  
> list of improvements that I'm planning to implement for  
> SessionSimulatorTest:
>
> - Use a CyclicBarrier to control read/write tasks start when all  
> threads are already up and running. Otherwise some taks could have  
> head start and begin before other tasks due to scheduling and this  
> could affect the results that we see.

+1

> - When measuring throughpuyt, we get more accurate measures by  
> timing the entire run and then dividing it by the number of  
> operations to get a per operation time. So, I'd suggest doing this  
> rather than measuring individual ops time. We can do this very  
> easily using CyclicBarriers as shown in JCIP.

True, but the problem with this is that all fwk overhead will be  
included in the results - such as time to generate random data, etc.

> - Currently, SessionSimulatorTest does not use the result of  
> cache.get() which is a bit unrealistic and could lead to the VM  
> optimizing the return path of the value. A recommended technique for  
> getting around this is recommended in the book.

+1

> - Finally, before warmup begins, we should add a check that the view  
> of the cluster is formed of the number of nodes that we're testing.

Isn't this already there?  Mircea?

> - Add an option to indicate the stress test timeout (currently  
> defaulting to 10 mins). This could be handy when profiling which  
> leads to longer execution times.

+1

Cheers
Manik


> Thoughts?
> -- 
> Galder Zamarreño
> Sr. Software Maintenance Engineer
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org