[infinispan-dev] Infinispan Benchmarks

Manik Surtani manik at jboss.org
Tue Nov 24 05:18:51 EST 2009


Did we ever get to the bottom of this?

On 19 Nov 2009, at 07:36, Bela Ban wrote:

> I looked at your test and noticed you're using 
> Object{Output,Input}Streams. These are very inefficient !
> 
> I suggest use simple data types for now, e.g. only ints for keys and 
> values. This way you could send a put as CMD | KEY | VAL, which is 3 
> ints. This would allow you to simply use a Data{Output,Input}Stream.
> 
> This is not the real world, I know, but we should focus on round trip 
> times and latency before we get into complex data type marshalling overhead.
> 
> 
> Michael Lawson (mshindo) wrote:
>> We have rejected the possibility of the problem being related to JGroups, as
>> when running then same configuration locally (not on the amazon e2).
>> 
>> *Let me outline the testing more specifically:*
>> 
>> I have created a very simple socket client and server to communicate with
>> infinispan nodes. This provides a mechanism to connect, send get and insert
>> commands coupled with the required data to the targeted infinispan nodes.
>> These insertions and retrievals are then timed from the client. As it stands
>> this system works perfectly in a local environment on my own network.
>> However as soon we attempt to test on the amazon e2 cloud, which is required
>> for benchmarking against other products, the retrieval times jump from under
>> a millisecond to around 160ms dependent on the value size number of nodes in
>> the cluster.
>> 
>> The reason we are testing using this client -> server model is that we are
>> also testing concurrency, to see what happens when we send thousands of
>> requests from different sources.
>> 
>> I have used TCPPing both locally and on the amazon cloud (as multi-casting
>> is not allowed in this environment), and the results are exactly the same.
>> Perfect numbers locally, bad numbers remotely. This is proving to be quite a
>> mystery.
>> 
>> I have uploaded my client and server code online base code:
>> http://pastebin.org/54960.
>> 
>> Any clues ?
>> 
>> On Wed, Nov 18, 2009 at 4:34 PM, Michael Lawson (mshindo) <
>> michael at sphinix.com> wrote:
>> 
>> 
>>> Are there any official socket clients available?
>>> 
>>> 
>>> On Tue, Nov 17, 2009 at 11:40 PM, Manik Surtani <manik at jboss.org> wrote:
>>> 
>>> 
>>>> On 17 Nov 2009, at 04:54, Michael Lawson (mshindo) wrote:
>>>> 
>>>> The benchmarking in question is simple insertions and retrievals run via
>>>> sockets, these benchmarks return better results when run on a local machine,
>>>> however the testing in question is being done on the Amazon E2 cloud.
>>>> Running on the E2 was a problem in itself, but I followed the instructions
>>>> on a blog and used an xml file to configure the transport properties.
>>>> 
>>>> <config xmlns="urn:org:jgroups" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:schemaLocation="urn:org:jgroups file:schema/JGroups-2.8.xsd">
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>        <TCP bind_port="7800" />
>>>>        <TCPPING timeout="3000"
>>>>                 initial_hosts="${jgroups.tcpping.initial_hosts:10.209.166.79[7800],10.209.198.176[7800],10.208.199.223[7800],10.208.190.224[7800],10.208.70.112[7800]}"
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>                port_range="1"
>>>>                num_initial_members="3"/>
>>>>         <MERGE2 max_interval="30000"  min_interval="10000"/>
>>>>         <FD_SOCK/>
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>         <FD timeout="10000" max_tries="5" />
>>>>         <VERIFY_SUSPECT timeout="1500"  />
>>>>        <pbcast.NAKACK
>>>>                 use_mcast_xmit="false" gc_lag="0"
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>                 retransmit_timeout="300,600,1200,2400,4800"
>>>>                discard_delivered_msgs="true"/>
>>>>        <UNICAST timeout="300,600,1200" />
>>>>        <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"  max_bytes="400000"/>
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>         <pbcast.GMS print_local_addr="true" join_timeout="3000"   view_bundling="true"/>
>>>>        <FC max_credits="2000000"  min_threshold="0.10"/>
>>>>        <FRAG2 frag_size="60000"  />
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>        <pbcast.STREAMING_STATE_TRANSFER/>
>>>> </config>
>>>> 
>>>> I have a theory, that perhaps the introduction of TCPPING in the jgroups
>>>> file is resulting in some form of polling before the actual get request is
>>>> processed and returned. Could this be the case ?
>>>> 
>>>> 
>>>> It could be - JGroups also has an experimental protocol called S3_PING
>>>> which could help.
>>>> 
>>>> 
>>>> http://javagroups.cvs.sourceforge.net/viewvc/javagroups/JGroups/src/org/jgroups/protocols/S3_PING.java?revision=1.2&view=markup
>>>> 
>>>> Another approach for discovery in an EC2 environment is to use a
>>>> GossipRouter, but I'd give S3_PING a try first.
>>>> 
>>>> Cheers
>>>> Manik
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Tue, Nov 17, 2009 at 12:03 AM, Manik Surtani <manik at jboss.org> wrote:
>>>> 
>>>> 
>>>>> Hi Michael
>>>>> 
>>>>> Could you please detail your benchmark test a bit more?  We have done
>>>>> some internal benchmarks as well and things do look significantly different.
>>>>> Could you also tell us which version you have been benchmarking?  We've
>>>>> made some significant changes to DIST between CR1 and CR2 with regards to
>>>>> performance.
>>>>> 
>>>>> FYI, we use the CacheBenchFwk [1] to help benchmark stuff; you may find
>>>>> this useful too.
>>>>> 
>>>>> Cheers
>>>>> Manik
>>>>> 
>>>>> [1] http://cachebenchfwk.sourceforge.net
>>>>> 
>>>>> 
>>>>> On 15 Nov 2009, at 22:00, Michael Lawson (mshindo) wrote:
>>>>> 
>>>>> 
>>>>>> Hi,
>>>>>> I have been performing some benchmark testing on Infinispan Running in
>>>>>> 
>>>>> Distributed mode, with some unexpected results.
>>>>> 
>>>>>> For an insertion with a Key size of 100 Bytes, and Value size 100
>>>>>> 
>>>>> Bytes, the insertion time was 0.13ms and retrieval was 128.06ms.
>>>>> 
>>>>>> Communication with the infinispan nodes is being done via a socket
>>>>>> 
>>>>> interface, using standard java serialization.
>>>>> 
>>>>>> The retrieval time is consistently high in comparison to other systems,
>>>>>> 
>>>>> and I am wondering whether there are some other benchmark reports floating
>>>>> around that I can compare results with.
>>>>> 
>>>>>> --
>>>>>> Michael Lawson
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> 
>>>>> --
>>>>> Manik Surtani
>>>>> manik at jboss.org
>>>>> Lead, Infinispan
>>>>> Lead, JBoss Cache
>>>>> http://www.infinispan.org
>>>>> http://www.jbosscache.org
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> 
>>>> 
>>>> --
>>>> Michael Lawson
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>> 
>>> --
>>> Michael Lawson (mshindo)
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> ------------------------------------------------------------------------
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> -- 
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org








More information about the infinispan-dev mailing list