[jboss-user] [JBoss Cache: Core Edition] - Stress Tests

[lovelyliatroim]

Ok I have been doing a bit of stress testing on my side. I have a scenario where my client has asked me to see if it is possible. Scenario is as follows, they current have at peak time 1500 pushes/writes a second going on in there DB. This 1500 writes a occurs on a data size of about 300,000 records. Due to re-design in their architecture they want to move this feed out of their DB.

Ive been asked to see if it is feasible to push it into there cache.

So following is used in the simulation Tomcat 6.0.13,JDK 1.6.0_06 and Jboss Cache 2.1.0

I created a local cache with 300,000 records. Code looks like so 



  | public void handleRequest(HashMap params, OutputStream out) {
  | 		// TODO Auto-generated method stub
  | 		//Create the cache on the first call and load it with records
  | 		if(cache == null){
  | 			cache = CacheManager.createCache("RealTime", "realtime-cache.xml");
  | 			for(int i = 0; i < noOfRecords;i++){
  | 				Map record = createQuoteMap();
  | 				String path = "/"+ i+ "/" + i +"/" +i ;
  | 				Fqn nodePath = Fqn.fromString(path);
  | 				cache.put(nodePath, "item", record);
  | 				if(i%500 == 0){
  | 					log.debug("Currently at Index"+i);
  | 				}
  | 			}
  | 			log.debug("Set up cache with "+noOfRecords +" records in the cache");
  | 
  | 		}
  | 		
  | 		requestCounter++;
  | 		
  | 		String action= RequestUtil.getString("action", params);
  | 		int index = generator.nextInt(noOfRecords);
  | 		String path = "/"+ index+ "/" + index +"/" + index ;
  | 		Fqn nodePath = Fqn.fromString(path);
  | 		
  | 		try {
  | 			if(action == null || action.compareTo("add") == 0){
  | 				Map record = createQuoteMap();
  | 				cache.put(path, "item", record);
  | 				writeCounter++;
  | 			}else{
  | 				//read operation
  | 				Map record = (Map)cache.get(nodePath, "item");
  | 				path = path +"\n";
  | 				out.write(path.getBytes());
  | 				DebugUtil.dumpMap(record, new PrintStream(out));
  | 				out.close();
  | 				readCounter++;
  | 			}
  | 			
  | 			if(requestCounter%1000 == 0){
  | 				log.debug("Read = "+readCounter + " Write = "+writeCounter);
  | 			}
  | 		} catch (IOException e) {
  | 			// TODO Auto-generated catch block
  | 			e.printStackTrace();
  | 		}
  | 	}
  | 

JMeter was set up to generate requests. Tests carried out on my local machine, Xp professional,2g ram, VM given 1G. VM is warmed up before recording begins.

Results are like so

 Test Case  Duration  No of Read Clients  No of Write/Update Threads  No of Reads  No of Updates/Writes  Throughput Reads (sec)  Throughput Updates/Writes(sec)  Isolation Level  Comments 

 Reading and Writing  1 Hour  10  10  1,889,946  1,754,769  525/sec  487.4/sec  NONE  39% error recorded in JMeter for updates and 36% error recorded as well for reads.Heap size between 100-175MB 

 Reading and Writing  1 Hour  10  10  5,741,899  5,974,218  1,594/sec  1,659/sec  READ_UNCOMMITTED  Heap size in between 300MB and 400MB range during processing. 0% error on both reading and writing. 

 Reading and Writing  30 Mins  10  10  2,940,466  3,046,956  1633.56/sec  1692.70/sec  READ_UNCOMMITTED  Heap size in between 300MB and 450MB range during processing. 0% error on both reading and writing. 

 Reading and Writing  30 Mins  10  10  728,026  743353  404.45/sec  412.95/sec  NONE  Heap size in between 100MB and 175MB range during processing. 44% error on reads and 43% on the writes. 

 - 
 Writing  30 mins  -  10  -  1,151,919  -  640/sec  NONE  56% error reported during run 

 Writing  30 mins  -  20  -  1,047,963  -  582.2/sec  NONE  63% error reported during run 

 Writing  30 mins  -  10  -  1,928,110  -  1071/sec  READ_UNCOMMITTED  21% error reported during run. Heap size between 300-400MB during run. 

 - 
 Reading  30 mins  10  -  5,670,148  -  3150.1/sec  -  NONE  0% error reported during run 

 Reading  30 mins  10  -  5,443,498  -  3018/sec/sec  -  READ_UNCOMMITTED  0% error reported during run. Heap size ranges from 280-400MB 

 Reading  30 mins  20  -  5,313,351  -  2951.8/sec  -  NONE  0% error reported during run 



Ok figures are taken from JMeters Summary report
http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Summary_Report.

Things that struck me are this
1. heap size difference, just by having a different isolation level from NONE to READ_UNCOMMITTED. I have seen this https://jira.jboss.org/jira/browse/JBCACHE-1383 and maybe that is playing a factor in this case. Will download latest CR version and see if it makes a difference. Heap Size when cache loaded with records in NONE mode was 100MB, when loaded in "READ_UNCOMIITED" 225MB. I possibly would have expected abit more in memory but not that much,its practically bigger than whats put in the cache.
2. Error % which is logged by JMeter, i tried to find their definition of what is classed as an error, unfortunately no such luck, best i could find was this definition " "Error % - Percent of requests with errors". So i will assume that this is possibly timeouts, maybe cache taking to long to write/read.
3. I would have thought that having an ISOLATION level of NONE would turn out to be quicker but apparently not. What goes on behind the scenes when its set to "NONE"?.
4. What I also found interesting was when in "READ_UNCOMMITTED" mode for just doing writes, I got 21% error back. Possibly contention to write to the same node?

Just thought Id post it, any feedback is welcome, any tips to improve throughput??  

Thanks,
LL
 


View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4166155#4166155

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4166155