[jbosscache-dev] Re: JBossCache Developments

[Manik Surtani]

Hi Mikko.

Good to hear from you!

I've cc'd the jbosscache-dev list as well, in case others are  
interested in following.

On 21 Mar 2007, at 21:31, Mikko Juhani Pitkaenen wrote:

>
> <SNIP />

> First, I took the CacheBenchFwk that Manik announced in the
> list. It was very easy to get started, I only run the basic
> test on three nodes.

Yes, this is something I started a while back, the goal being that  
most benchmarks are pretty useless and are "tweaked" to look good by  
using a specific usage pattern, configuration and hardware.  To make  
benchmark comparisons more realistic, I allow people to write their  
own "test", a simple use case to reflect real-live cache usage  
(transactions, #puts vs #gets, type and size of payload, etc) and to  
plug in different cache configurations or even engines (JBC 1.4 vs  
JBC 2.0 vs EHCache, for example)    It still has a way to go though.
>
> I have too different "clusters" to use. They are in fact
> classrooms of PCs and during nighttime they are rarely used
> for anything. When I have the benchmarks working on these
> I can try to get my hands on a real cluster too.
>
> First classroom:
>
> 30 × Ubuntu Linux / Intel Celeron 2.4 Ghz / 1 GB RAM
>
> Another Classroom:
>
> 49 × Ubuntu Linux /  Intel Celeron 2,2 GHz / 512 MB RAM
>
> With these it is possible to test already with pretty
> decent node count. I will find out about the network
> interconnect also, and Java is for now:
>
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
> Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)
>

The node count is useful, although like you said knowing what sort of  
network they are on is probably even more useful.  :-)  The JDK used  
is fine as far as I am concerned.
>
> I am bit of a fan of performance testing, and it would be
> nice to integrate some kind of graphing tool (maybe gnuplot)
> to this too. In this environment it is pretty easy to start/kill
> processes on different machines also through secure shell, so
> the test could be quite easily automated to run each benchmark
> with different number of slaves started.
>

Reporting is pretty easy with the framework as well.  Internally, a  
TestResult object represents the measurements, and reports are  
created using the ReportGenerator interface.  So far I have a  
CSVReportGenerator but others like a GraphicalReportGenerator which  
uses gnuplot should be simple to implement.  (I just used the CSV  
reports and plugged them into Excel and drew graphs there!)

One of the other things I'd like to add to the test result is  
throughput.  Calculations are made on various statistical analyses on  
time taken for puts and gets (mean, median, mode, max, min, standard  
deviation) but calculating throughput (repplication speed in bytes/ 
sec) as well as requests per second would be useful metrics to capture.

> Second, I have checked out the latest development branch to start
> thinking the architecture for the striping cache we discussed about.
> To sketch out a good plan I still need to know the code better.

Sure, this would be a separate thread anyway.

Cheers,
--
Manik Surtani

Lead, JBoss Cache
JBoss, a division of Red Hat

Email: manik at jboss.org
Telephone: +44 7786 702 706
MSN: manik at surtani.org
Yahoo/AIM/Skype: maniksurtani