[Hawkular-dev] RxJava2 preliminary testing

[Michael Burman]

Hi,

I did yesterday evening and today some testing on how using RxJava2 
would benefit us (I'm expecting more from RxJava 2.1 actually, since it 
has some enhanced parallelism features which we might benefit from).

Short notes from RxJava2 migration, it's more painful than I assumed. 
The code changes can be small in terms of lines of code changed, but 
almost every method has had their signature or behavior changed. So at 
least I've had to read the documentation all the time when doing things 
and trying to unlearn what I've done in the RxJava1.

And all this comes with a backwards compatibility pressure for Java 6 
(so you can't benefit from many Java 8 advantages). Reactive-Commons / 
Reactor have started from Java 8 to provide cleaner implementation. Grr.

I wrote a simple write path modification in the PR #762 (metrics) that 
writes Gauges using RxJava2 ported micro-batching feature. There's still 
some RxJavaInterOp use in it, so that might slow down the performance a 
little bit. However, it is possible to merge these two codes. There are 
also some other optimizations I think could be worth it.

I'd advice against it though, reading gets quite complex. I would almost 
suggest that we would do the MetricsServiceImpl/DataAccessImpl merging 
by rewriting small parts at a time in the new class with RxJava2 and 
make that call the old code with RxJavaInterOp. That way we could move 
slowly to the newer codebase.

I fixed the JMH-benchmarks (as they're not compiled in our CI and were 
actually broken by some other PRs) and ran some tests. These are the 
tests that measure only the metrics-core-service performance and do not 
touch the REST-interface (or Wildfly) at all, thus giving better 
comparison in how our internal changes behave.

What I'm seeing is around 20-30% difference in performance when writing 
gauges this way. So this should offset some of the issues we saw when we 
improved error handling (which caused performance degradation). I did 
ran into the HWKMETRICS-542 (BusyPoolException) so the tests were run 
with 1024 connections.

I'll continue next week some more testing, but at the same time I proved 
that the micro-batching features do improve performance in the internal 
processing, especially when there's small amount of writers to a single 
node. But testing those features could probably benefit from more 
benchmark tests without WIldfly (which takes so much processing power 
that most performance improvements can't be measured correctly anymore).

    - Micke