[infinispan-dev] Improvements for SessionSimulatorTest in benchmark framework

[Galder Zamarreno]

Manik Surtani wrote:
> 
> On 2 Jun 2009, at 11:40, Galder Zamarreno wrote:
> 
>> Hi all,
>>
>> Over the weekend, I've been reading the JCIP and I've come up with a 
>> list of improvements that I'm planning to implement for 
>> SessionSimulatorTest:
>>
>> - Use a CyclicBarrier to control read/write tasks start when all 
>> threads are already up and running. Otherwise some taks could have 
>> head start and begin before other tasks due to scheduling and this 
>> could affect the results that we see.
> 
> +1
> 
>> - When measuring throughpuyt, we get more accurate measures by timing 
>> the entire run and then dividing it by the number of operations to get 
>> a per operation time. So, I'd suggest doing this rather than measuring 
>> individual ops time. We can do this very easily using CyclicBarriers 
>> as shown in JCIP.
> 
> True, but the problem with this is that all fwk overhead will be 
> included in the results - such as time to generate random data, etc.

No, it shouldn't. If generating random data is put before 
barrier.await() in the task, the start time will only be computed after 
the barrier.await() releases, so generating random data won't be 
included. See

http://book.javanb.com/java-concurrency-in-Practice/ch12lev1sec2.html

and Producer/Consumer classes in 
http://book.javanb.com/java-concurrency-in-Practice/ch12lev1sec1.html 
(they're like our Read and Write tasks,)

> 
>> - Currently, SessionSimulatorTest does not use the result of 
>> cache.get() which is a bit unrealistic and could lead to the VM 
>> optimizing the return path of the value. A recommended technique for 
>> getting around this is recommended in the book.
> 
> +1
> 
>> - Finally, before warmup begins, we should add a check that the view 
>> of the cluster is formed of the number of nodes that we're testing.
> 
> Isn't this already there?  Mircea?

It checks that the number of nodes is there connecting to them to colate 
the results but does not check the cluster view itself. That's how I was 
able to run the tests even though each node was a single node on its own.

> 
>> - Add an option to indicate the stress test timeout (currently 
>> defaulting to 10 mins). This could be handy when profiling which leads 
>> to longer execution times.
> 
> +1
> 
> Cheers
> Manik
> 
> 
>> Thoughts?
>> -- 
>> Galder Zamarreño
>> Sr. Software Maintenance Engineer
>> JBoss, a division of Red Hat
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> -- 
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 

-- 
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat