Re: [infinispan-dev] Performance gap between different value sizes and between key loactions

-----Original Message----- From: infinispan-dev-bounces(a)lists.jboss.org [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Dan Berindei Sent: Monday, December 15, 2014 11:44 PM To: infinispan -Dev List Subject: Re: [infinispan-dev] Performance gap between different value sizes and between key loactions JR, could you share your test, or at least the configuration you used and what key/value types you used? Like Radim said, in your 1+0 scenario with storeAsBinary disabled and no cache store attached, I would expect the latency to be exactly the same for all value sizes. Cheers Dan On Mon, Dec 15, 2014 at 3:34 PM, Radim Vansa <rvansa(a)redhat.com&gt; wrote: > Hi JR, > > thanks for those findings! I was benchmarking the dependency of > achieved throughput based on entry size in the past, and I found the > sweet spot on 8k values (likely because our machines had 9k MTU). > Regrettably, we were focusing on throughput rather than on latency. > > I think that the increased latency could be on the account of: > a) marshalling - this is the top suspect > b) when receiving the data from network (in JGroups), those are copied > from the socket to buffer > c) general GC activity - with larger data flow you're about to trigger > GC sooner > > Though, I am quite surprised by such linear scaling, usually RPC > latency or waiting for locks is the villain. Unless you set in cache > configuration to storeAsBinary, Infinispan treats values as references > and there should be no overhead involved. > > Could you set up sampling mode profiler and check what it reports? All > the above are just slightly educated guesses. > > Radim > > On 12/15/2014 01:54 PM, linjunru wrote: >> >> Hi, all: >> >> I have tested infinispan in distributed mode in terms of latency of >> put(k,v) operation. The own_num is 1 and the key we put/write locates >> in the same node as the put operation occurs(In the table,“1+0” >> represents this scenario), the results indicates that the latency >> increases as the size of the value increases. However the increments >> seem to be a little “unreasonable” to me, because the bandwidth of >> the memory system is quite huge, and the number of keys (10000) >> remains the same during the experiment. So, here is the questions: >> which operations inside infinspan have strong relatives with the size >> of value, and why they costs so much as the size increases? >> >> We have also tested infinispan in the scenario which the key and the >> put/write(key,value) operation reside in different nodes(we noted it >> as “0+1”). Compared with “1+0”, “0+1” triggers network >> communications, however, the network latency is much smaller compared >> to the performance gas between the two scenarios. Why this situation happens? >> For example, with a 25K bytes ping packet, the RTT is about 0.713ms >> while performance gas between the two scenarios is about 8.4ms，what >> operations inside infinispan used the other 7.6ms? >> >> UDP is utilized as the transport protocol, the infinispan version we >> used is 7.0 and there are 4 nodes in the cluster, each has 10000 >> keys, all of them have memory bigger than 32G, and all of them have >> xeon cpu >> e5-2407 x2. >> >> Value size >> >> >> >> 250B( us) >> >> >> >> 2.5K( us) >> >> >> >> 25k(us) >> >> >> >> 250k(us) >> >> >> >> 2.5M(us) >> >> >> >> 25M(us) >> >> 1+0 >> >> >> >> 463 >> >> >> >> 726 >> >> >> >> 3 236 >> >> >> >> 26 560 >> >> >> >> 354 454 >> >> >> >> 3 979 830 >> >> 0+1 >> >> >> >> 1 807 >> >> >> >> 2 829 >> >> >> >> 11 635 >> >> >> >> 87 540 >> >> >> >> 1 035 133 >> >> >> >> 11 653 389 >> >> Thanks! >> >> Best Regards, >> >> JR >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa <rvansa(a)redhat.com&gt; > JBoss DataGrid QA > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev