Hello,
I tried to find optimal message size for POST requests which will result into the highest
total throughput.
I tried 1, 100, 500 and 5000 metrics per request.
Test:
- h-metrics VM: 4 CPU cores, 8GB memory (gradually increasing heap size for WF in each
test run - 1GB, 2GB, 4GB, 6GB, 8GB)
- cassandra node VM: 2 CPU cores, 4GB memory
Summary:
- small messages (1 metrics/request) were the worst for all test runs -> total
throughput ~ 4500 metrics/sec
- medium messages (100 metrics/request) were performing the best almost for all test runs
-> total throughput ~ 8600 -- 9000 metrics/sec
- bigger messages (500 and 5000 metrics/request) were significantly worst than medium
messages when having low heap size (1GB)
- bigger messages were worst than medium even for bigger heap sizes but it was getting
close
- big messages (5000 metrics/request) overloaded h-metrics when having only 1GB heap ->
Full GC all the time
- when adding additional cassandra node and having sufficient heap size the bigger
messages were performing better -> with sufficient heap size (4GB) and cassandra
cluster the bigger messages the better performance (measured only up to 5000, I'm not
sure if it make sense to try even bigger messages)
Details:
4 CPU cores
1GB for WF
msg size=1
4400 req/sec -> total throughput = 4400
avg-cpu: %user %nice %system %iowait %steal %idle
68.47 0.00 12.39 0.00 0.67 18.46
msg size=100
87 req/sec -> total throughput = 8700
avg-cpu: %user %nice %system %iowait %steal %idle
24.94 0.00 11.15 0.05 0.28 63.58
msg size=500
13 req/sec -> total throughput = 6500
avg-cpu: %user %nice %system %iowait %steal %idle
41.80 0.00 7.19 0.08 0.15 50.78
msg size=5000
0.879 req/sec -> total throughput = 4395
- GC all the time
avg-cpu: %user %nice %system %iowait %steal %idle
66.35 0.00 4.08 0.00 0.08 29.50
===================================
2GB for WF
msg size=1
4574 req/sec -> total throughput = 4574
avg-cpu: %user %nice %system %iowait %steal %idle
67.84 0.00 12.67 0.05 0.87 18.57
msg size=100
87 req/sec -> total throughput = 8700
- GC each 5-6 minutes
avg-cpu: %user %nice %system %iowait %steal %idle
26.99 0.00 10.25 0.13 0.21 62.43
msg size=500
16 req/sec -> total throughput = 8000
- GC each 1 min
avg-cpu: %user %nice %system %iowait %steal %idle
28.48 0.00 8.22 0.15 0.59 62.55
msg size=5000
1.432 req/sec -> total throughput = 7160
- GC each 30s
avg-cpu: %user %nice %system %iowait %steal %idle
36.44 0.00 10.04 0.18 0.08 53.27
===================================
4GB for WF
msg size=1
4498 req/sec -> total throughput = 4498
avg-cpu: %user %nice %system %iowait %steal %idle
63.28 0.00 13.03 0.03 0.58 23.10
msg size=100
90.79 req/sec -> total throughput = 9079
- GC after > 6 minutes
avg-cpu: %user %nice %system %iowait %steal %idle
25.86 0.00 10.44 0.08 0.26 63.37
msg size=500
16.07 req/sec -> total throughput = 8035
- GC after > 6 minutes
avg-cpu: %user %nice %system %iowait %steal %idle
23.73 0.00 8.18 0.13 0.46 67.50
msg size=5000
1.545 req/sec -> total throughput = 7725
- GC after > 6 minutes
avg-cpu: %user %nice %system %iowait %steal %idle
31.35 0.00 10.20 0.05 0.16 58.24
===================================
6GB for WF
msg size=1
4556 req/sec -> total throughput = 4556
avg-cpu: %user %nice %system %iowait %steal %idle
67.59 0.00 12.24 0.15 0.50 19.52
msg size=100
86 req/sec -> total throughput = 8600
avg-cpu: %user %nice %system %iowait %steal %idle
27.40 0.00 10.36 0.13 0.28 61.83
msg size=500
18 req/sec -> total throughput = 9000
avg-cpu: %user %nice %system %iowait %steal %idle
29.30 0.00 8.49 0.18 0.26 61.77
msg size=5000
1.45 req/sec -> total throughput = 7250
avg-cpu: %user %nice %system %iowait %steal %idle
32.26 0.00 11.61 0.34 0.10 55.69
===================================
8GB for WF
msg size=1
4718 req/sec -> total throughput = 4718
avg-cpu: %user %nice %system %iowait %steal %idle
44.19 0.00 17.05 0.33 0.22 38.22
msg size=100
82.27 req/sec -> total throughput = 8227
avg-cpu: %user %nice %system %iowait %steal %idle
30.77 0.00 10.95 0.08 0.13 58.07
msg size=500
17.86 req/sec -> total throughput = 7725
avg-cpu: %user %nice %system %iowait %steal %idle
52.56 20.32 11.16 0.20 0.00 15.76
msg size=5000
failed
===================================
2GB for WF + 2 cassandra nodes
msg size=1
total throughput = 6741
msg size=100
total throughput = 12900
msg size=500
total throughput = 11250
msg size=5000
total throughput = 13100
===================================
4GB for WF + 2 cassandra nodes
msg size=1
total throughput = 6533
msg size=100
total throughput = 12280
msg size=500
total throughput = 12650
msg size=5000
total throughput = 15950
Filip
Show replies by date