Re: [infinispan-dev] Improvements for SessionSimulatorTest in benchmark framework
Manik Surtani wrote:
On 2 Jun 2009, at 11:40, Galder Zamarreno wrote:
> Hi all,
>
> Over the weekend, I've been reading the JCIP and I've come up with a
> list of improvements that I'm planning to implement for
> SessionSimulatorTest:
>
> - Use a CyclicBarrier to control read/write tasks start when all
> threads are already up and running. Otherwise some taks could have
> head start and begin before other tasks due to scheduling and this
> could affect the results that we see.
+1
> - When measuring throughpuyt, we get more accurate measures by timing
> the entire run and then dividing it by the number of operations to get
> a per operation time. So, I'd suggest doing this rather than measuring
> individual ops time. We can do this very easily using CyclicBarriers
> as shown in JCIP.
True, but the problem with this is that all fwk overhead will be
included in the results - such as time to generate random data, etc.
No, it shouldn't. If generating random data is put before
barrier.await() in the task, the start time will only be computed after
the barrier.await() releases, so generating random data won't be
included. See
http://book.javanb.com/java-concurrency-in-Practice/ch12lev1sec2.html
and Producer/Consumer classes in
http://book.javanb.com/java-concurrency-in-Practice/ch12lev1sec1.html
(they're like our Read and Write tasks,)
> - Currently, SessionSimulatorTest does not use the result of
> cache.get() which is a bit unrealistic and could lead to the VM
> optimizing the return path of the value. A recommended technique for
> getting around this is recommended in the book.
+1
> - Finally, before warmup begins, we should add a check that the view
> of the cluster is formed of the number of nodes that we're testing.
Isn't this already there? Mircea?
It checks that the number of nodes is there connecting to them to colate
the results but does not check the cluster view itself. That's how I was
able to run the tests even though each node was a single node on its own.
> - Add an option to indicate the stress test timeout (currently
> defaulting to 10 mins). This could be handy when profiling which leads
> to longer execution times.
+1
Cheers
Manik
> Thoughts?
> --
> Galder Zamarreño
> Sr. Software Maintenance Engineer
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
--
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat