[jbosscache-dev] Re: JBossCache Developments
Hi Mikko.
Good to hear from you!
I've cc'd the jbosscache-dev list as well, in case others are
interested in following.
On 21 Mar 2007, at 21:31, Mikko Juhani Pitkaenen wrote:
<SNIP />
First, I took the CacheBenchFwk that Manik announced in the
list. It was very easy to get started, I only run the basic
test on three nodes.
Yes, this is something I started a while back, the goal being that
most benchmarks are pretty useless and are "tweaked" to look good by
using a specific usage pattern, configuration and hardware. To make
benchmark comparisons more realistic, I allow people to write their
own "test", a simple use case to reflect real-live cache usage
(transactions, #puts vs #gets, type and size of payload, etc) and to
plug in different cache configurations or even engines (JBC 1.4 vs
JBC 2.0 vs EHCache, for example) It still has a way to go though.
I have too different "clusters" to use. They are in fact
classrooms of PCs and during nighttime they are rarely used
for anything. When I have the benchmarks working on these
I can try to get my hands on a real cluster too.
First classroom:
30 × Ubuntu Linux / Intel Celeron 2.4 Ghz / 1 GB RAM
Another Classroom:
49 × Ubuntu Linux / Intel Celeron 2,2 GHz / 512 MB RAM
With these it is possible to test already with pretty
decent node count. I will find out about the network
interconnect also, and Java is for now:
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)
The node count is useful, although like you said knowing what sort of
network they are on is probably even more useful. :-) The JDK used
is fine as far as I am concerned.
I am bit of a fan of performance testing, and it would be
nice to integrate some kind of graphing tool (maybe gnuplot)
to this too. In this environment it is pretty easy to start/kill
processes on different machines also through secure shell, so
the test could be quite easily automated to run each benchmark
with different number of slaves started.
Reporting is pretty easy with the framework as well. Internally, a
TestResult object represents the measurements, and reports are
created using the ReportGenerator interface. So far I have a
CSVReportGenerator but others like a GraphicalReportGenerator which
uses gnuplot should be simple to implement. (I just used the CSV
reports and plugged them into Excel and drew graphs there!)
One of the other things I'd like to add to the test result is
throughput. Calculations are made on various statistical analyses on
time taken for puts and gets (mean, median, mode, max, min, standard
deviation) but calculating throughput (repplication speed in bytes/
sec) as well as requests per second would be useful metrics to capture.
Second, I have checked out the latest development branch to start
thinking the architecture for the striping cache we discussed about.
To sketch out a good plan I still need to know the code better.
Sure, this would be a separate thread anyway.
Cheers,
--
Manik Surtani
Lead, JBoss Cache
JBoss, a division of Red Hat
Email: manik(a)jboss.org
Telephone: +44 7786 702 706
MSN: manik(a)surtani.org
Yahoo/AIM/Skype: maniksurtani