[Hawkular-dev] Performance Tests of Different Hawkular Inventory Backends

[Lukas Krejci]

Hi all,

lately I've become really dissatisfied with how Inventory performed and 
semi-publicly blamed Titan for that (because that was what looked like 
the cause of all world's problems in my then uneducated eyes ;) ).

I decided to do some performance comparisons. Because we didn't want 
Hawkular to ship with 2 different NoSQL backends (C* for metrics and 
whatever else for Inventory), I chose an RDBMS as a good conservative 
alternative (because people, IMHO, are still more comfortable dealing 
with an RDBMS than with NoSQL databases).

Currently, inventory is written against the graph DSL called Gremlin 
(from Tinkerpop 2.6.0). Fortunately, there exists a "toy" SQL backend 
for Tinkerpop 2 that we could try and see if it performed any good 
(which would frankly be surprising, given the fact it stores the graph 
data rather naively). With some luck, no code would have to be changed 
on our side to see the results.

We had no such luck.

Making the inventory run with the SQL backend was literally a day worth 
of work (if that) and the first preliminary tests showed that Inventory 
with Postgres backend performed much much better that Titan with 
embedded Cassandra. But the tests also uncovered some problems with the 
way Inventory code handled transactions.

Fast forward 3 weeks and see large parts of Hawkular inventory updated 
to correctly handle transactions. Now a single call to Inventory really 
results in at most 1 transaction in the backend.

So, I went and re-ran the tests. Also, I refrained from using embedded 
Cassandra and instead use a locally running 2-node cluster.

The results caught me by surprise. Not so much that the naive SQL 
backend didn't perform particularly well, but the difference between the 
performance of Titan before and after the transaction handling fixes.

To not keep you waiting any longer for the results: Titan + C* is the 
winner.

For nice charts that include comparison to the old misbehaving impl, see:
https://dashboards.ly/ua-tALzrY9rEoRBXvsLXbZJHT

Cheers,

-- 
Lukas Krejci