4:04 a.m.
Hi,
I did yesterday evening and today some testing on how using RxJava2
would benefit us (I'm expecting more from RxJava 2.1 actually, since it
has some enhanced parallelism features which we might benefit from).
Short notes from RxJava2 migration, it's more painful than I assumed.
The code changes can be small in terms of lines of code changed, but
almost every method has had their signature or behavior changed. So at
least I've had to read the documentation all the time when doing things
and trying to unlearn what I've done in the RxJava1.
And all this comes with a backwards compatibility pressure for Java 6
(so you can't benefit from many Java 8 advantages). Reactive-Commons /
Reactor have started from Java 8 to provide cleaner implementation. Grr.
I wrote a simple write path modification in the PR #762 (metrics) that
writes Gauges using RxJava2 ported micro-batching feature. There's still
some RxJavaInterOp use in it, so that might slow down the performance a
little bit. However, it is possible to merge these two codes. There are
also some other optimizations I think could be worth it.
I'd advice against it though, reading gets quite complex. I would almost
suggest that we would do the MetricsServiceImpl/DataAccessImpl merging
by rewriting small parts at a time in the new class with RxJava2 and
make that call the old code with RxJavaInterOp. That way we could move
slowly to the newer codebase.
I fixed the JMH-benchmarks (as they're not compiled in our CI and were
actually broken by some other PRs) and ran some tests. These are the
tests that measure only the metrics-core-service performance and do not
touch the REST-interface (or Wildfly) at all, thus giving better
comparison in how our internal changes behave.
What I'm seeing is around 20-30% difference in performance when writing
gauges this way. So this should offset some of the issues we saw when we
improved error handling (which caused performance degradation). I did
ran into the HWKMETRICS-542 (BusyPoolException) so the tests were run
with 1024 connections.
I'll continue next week some more testing, but at the same time I proved
that the micro-batching features do improve performance in the internal
processing, especially when there's small amount of writers to a single
node. But testing those features could probably benefit from more
benchmark tests without WIldfly (which takes so much processing power
that most performance improvements can't be measured correctly anymore).
- Micke