Re: [infinispan-dev] Improvements for SessionSimulatorTest in benchmark framework
Galder Zamarreno wrote:
Manik Surtani wrote:
>
> On 2 Jun 2009, at 11:40, Galder Zamarreno wrote:
>
>> Hi all,
>>
>> Over the weekend, I've been reading the JCIP and I've come up with a
>> list of improvements that I'm planning to implement for
>> SessionSimulatorTest:
>>
>> - Use a CyclicBarrier to control read/write tasks start when all
>> threads are already up and running. Otherwise some taks could have
>> head start and begin before other tasks due to scheduling and this
>> could affect the results that we see.
>
> +1
+1
>
>> - When measuring throughpuyt, we get more accurate measures by
>> timing the entire run and then dividing it by the number of
>> operations to get a per operation time. So, I'd suggest doing this
>> rather than measuring individual ops time. We can do this very
>> easily using CyclicBarriers as shown in JCIP.
>
> True, but the problem with this is that all fwk overhead will be
> included in the results - such as time to generate random data, etc.
No, it shouldn't. If generating random data is put before
barrier.await() in the task, the start time will only be computed
after the barrier.await() releases, so generating random data won't be
included. See
http://book.javanb.com/java-concurrency-in-Practice/ch12lev1sec2.html
and Producer/Consumer classes in
http://book.javanb.com/java-concurrency-in-Practice/ch12lev1sec1.html
(they're like our Read and Write tasks,)
>
>> - Currently, SessionSimulatorTest does not use the result of
>> cache.get() which is a bit unrealistic and could lead to the VM
>> optimizing the return path of the value. A recommended technique for
>> getting around this is recommended in the book.
>
> +1
>
>> - Finally, before warmup begins, we should add a check that the view
>> of the cluster is formed of the number of nodes that we're testing.
>
> Isn't this already there? Mircea?
It checks that the number of nodes is there connecting to them to
colate the results but does not check the cluster view itself. That's
how I was able to run the tests even though each node was a single
node on its own.
Hmm, it does a put on each node and expects other caches to
replicate on
the same node(at least one, if partialReplication is true). Each node
running on itself scenario should be catch by this test, as it only
counts the number of different nodes that replicated here, excluding
self. Only scenario it does not catch is when two partitions are created
- so that some replication exists. I haven't run through such a problem,
though. Re:checking against cluster view itslef, that is possible for
solutions build on top of jgroups (JBC, infinispan) but it would be more
difficult to implement for coherene, teracotta etc
>
>> - Add an option to indicate the stress test timeout (currently
>> defaulting to 10 mins). This could be handy when profiling which
>> leads to longer execution times.
>
> +1
>
> Cheers
> Manik
>
>
>> Thoughts?
>> --
>> Galder Zamarreño
>> Sr. Software Maintenance Engineer
>> JBoss, a division of Red Hat
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>