Hi JR,
thanks for those findings! I was benchmarking the dependency of achieved
throughput based on entry size in the past, and I found the sweet spot
on 8k values (likely because our machines had 9k MTU). Regrettably, we
were focusing on throughput rather than on latency.
I think that the increased latency could be on the account of:
a) marshalling - this is the top suspect
b) when receiving the data from network (in JGroups), those are copied
from the socket to buffer
c) general GC activity - with larger data flow you're about to trigger
GC sooner
Though, I am quite surprised by such linear scaling, usually RPC latency
or waiting for locks is the villain. Unless you set in cache
configuration to storeAsBinary, Infinispan treats values as references
and there should be no overhead involved.
Could you set up sampling mode profiler and check what it reports? All
the above are just slightly educated guesses.
Radim
On 12/15/2014 01:54 PM, linjunru wrote:
Hi, all:
I have tested infinispan in distributed mode in terms of latency of
put(k,v) operation. The own_num is 1 and the key we put/write locates
in the same node as the put operation occurs(In the table,“1+0”
represents this scenario), the results indicates that the latency
increases as the size of the value increases. However the increments
seem to be a little “unreasonable” to me, because the bandwidth of the
memory system is quite huge, and the number of keys (10000) remains
the same during the experiment. So, here is the questions: which
operations inside infinspan have strong relatives with the size of
value, and why they costs so much as the size increases?
We have also tested infinispan in the scenario which the key and the
put/write(key,value) operation reside in different nodes(we noted it
as “0+1”). Compared with “1+0”, “0+1” triggers network communications,
however, the network latency is much smaller compared to the
performance gas between the two scenarios. Why this situation happens?
For example, with a 25K bytes ping packet, the RTT is about 0.713ms
while performance gas between the two scenarios is about 8.4ms,what
operations inside infinispan used the other 7.6ms?
UDP is utilized as the transport protocol, the infinispan version we
used is 7.0 and there are 4 nodes in the cluster, each has 10000 keys,
all of them have memory bigger than 32G, and all of them have xeon cpu
e5-2407 x2.
Value size
250B( us)
2.5K( us)
25k(us)
250k(us)
2.5M(us)
25M(us)
1+0
463
726
3 236
26 560
354 454
3 979 830
0+1
1 807
2 829
11 635
87 540
1 035 133
11 653 389
Thanks!
Best Regards,
JR
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa(a)redhat.com>
JBoss DataGrid QA