[hibernate-dev] Hibernate Validator 6 benchmarks results

Hi, I ran some HV 6 benchmarks results this morning, following the profiling campaign I did last week and prior to our 6.0.0.Alpha2 release, and got some interesting results I thought worthwhile to share. Results are in ops/ms and are from our own JMH benchmarks. The higher, the better. == 5.4.1.Final Result "org.hibernate.validator.performance.simple.SimpleValidation.testSimpleBeanValidation": 556.190 ±(99.9%) 3.985 ops/ms [Average] (min, avg, max) = (537.875, 556.190, 570.387), stdev = 8.049 CI (99.9%): [552.205, 560.175] (assumes normal distribution) Result "org.hibernate.validator.performance.cascaded.CascadedValidation.testCascadedValidation": 288.429 ±(99.9%) 3.305 ops/ms [Average] (min, avg, max) = (271.979, 288.429, 308.106), stdev = 6.677 CI (99.9%): [285.124, 291.734] (assumes normal distribution) == Current master Result "org.hibernate.validator.performance.simple.SimpleValidation.testSimpleBeanValidation": 869.546 ±(99.9%) 14.734 ops/ms [Average] (min, avg, max) = (760.007, 869.546, 909.206), stdev = 29.763 CI (99.9%): [854.813, 884.280] (assumes normal distribution) Result "org.hibernate.validator.performance.cascaded.CascadedValidation.testCascadedValidation": 343.699 ±(99.9%) 2.077 ops/ms [Average] (min, avg, max) = (331.333, 343.699, 352.626), stdev = 4.196 CI (99.9%): [341.622, 345.776] (assumes normal distribution) == Analysis === SimpleValidation So, interestingly enough, current master is 56% faster than 5.4.1.Final on the simple validation case. This is all due to the fact that we now cache the ValueExtractor in the metadata whereas the value unwrapper was determined at validation time before. All the CPU goes here: https://github.com/hibernate/hibernate-validator/blob/5.4/engine/src/main... and especially on the typeResolver.resolve calls. This is mostly a structural improvement due to the new design of ValueExtractors (kudos Gunnar!). === CascadedValidation So, last week, HV 6 was 3 times slower (yes **3 times** slower) than HV 5.4 on the CascadedValidation benchmark. After my recent improvements, it's now 20% faster so it's quite conclusive too. I obtained these results mostly by limiting the number of reflection calls (again) and better caching of some metadata. == Conclusion So 2 conclusions mostly: - we are on the right track - we really need to be careful when doing reflection operations, they might look harmless at the time but they consume a lot of cycles Have a nice day! -- Guillaume