Hi Mikko. Good to hear from you! I've cc'd the jbosscache-dev list as well, in case others are interested in following. On 21 Mar 2007, at 21:31, Mikko Juhani Pitkaenen wrote: Yes, this is something I started a while back, the goal being that most benchmarks are pretty useless and are "tweaked" to look good by using a specific usage pattern, configuration and hardware. To make benchmark comparisons more realistic, I allow people to write their own "test", a simple use case to reflect real-live cache usage (transactions, #puts vs #gets, type and size of payload, etc) and to plug in different cache configurations or even engines (JBC 1.4 vs JBC 2.0 vs EHCache, for example) It still has a way to go though. The node count is useful, although like you said knowing what sort of network they are on is probably even more useful. :-) The JDK used is fine as far as I am concerned. Reporting is pretty easy with the framework as well. Internally, a TestResult object represents the measurements, and reports are created using the ReportGenerator interface. So far I have a CSVReportGenerator but others like a GraphicalReportGenerator which uses gnuplot should be simple to implement. (I just used the CSV reports and plugged them into Excel and drew graphs there!) One of the other things I'd like to add to the test result is throughput. Calculations are made on various statistical analyses on time taken for puts and gets (mean, median, mode, max, min, standard deviation) but calculating throughput (repplication speed in bytes/ sec) as well as requests per second would be useful metrics to capture. Sure, this would be a separate thread anyway. Cheers, -- Manik Surtani Lead, JBoss Cache JBoss, a division of Red Hat Email: manik(a)jboss.org Telephone: +44 7786 702 706 MSN: manik(a)surtani.org Yahoo/AIM/Skype: maniksurtani