<div dir="ltr">I discussed some more with Lukas and looked around a bit.<div><br></div><div>Titan is basically the link between TinkerPop (graph API) and Cassandra.</div><div><br></div><div>I may have jumped in conclusion but:</div><div> - Datastax (the company behind Cassandra) bought ThinkAurelius (the company behind Titan)</div><div> - Since then Datastax built a product "inspired by Titan", a graph DB with TinkerPop and Cassandra. TThe product is closed-source and completely targeted to Cassandra. Datastax has no real incentive to maintain Titian as it competes with their product and all engineers stopped contributing to it.</div><div> - Last release of Titan was in Sept 2015 (they used to release ~ every 3 months)</div><div> - While the community is relatively active, no Pull request was approved after June, and we don't know about any fork that is well maintained</div><div> - It's a fairly large and complex piece of code, too large and too narrowed for us to take over.</div><div><br></div><div>Conclusion: medium/long term Titan is no longer an option.</div><div><br></div><div>The other concern is that there is no good solution to store to Cassandra which is the only storage dependence for Hawkular Services today (and was an important requirement).</div><div><br></div><div>So we have to make a difficult choice here but we don't seem to have many options if we stick with TinkerPop at least...</div><div><br></div><div>According to Lukas, Sqlg is a good option for the following reasons:</div><div> - Performance</div><div> - Size/complexity</div><div> - The "community" is really small but the lead developer is responsive (and in the case he stops, it would be easier to fork and maintain).</div><div><br></div><div>The big drawback is that for production we would require Postgres (for non-prod or for Hawkular Services users who don't use the inventory service, we can use the embedded H2).</div><div><br></div><div>Thoughts ?</div><div><br></div><div>Thomas</div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 21, 2016 at 2:08 PM, Lukas Krejci <span dir="ltr"><<a href="mailto:lkrejci@redhat.com" target="_blank">lkrejci@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
to move inventory forward, we need to port it to Tinkerpop3 - a new(ish) and<br>
actively maintained version of the Tinkerpop graph API.<br>
<br>
Apart from the huge improvement in the API expressiveness and capabilities,<br>
the important thing is that it comes with a variety of backends, 2 of which<br>
are of particular interest to us ATM. The Titan backend (with Titan in version<br>
1.0) and SQL backend (using the sqlg library).<br>
<br>
The SQL backend is a much improved (yet still unfinished in terms of<br>
optimizations and some corner case features) version of the toy SQL backend<br>
for Tinkerpop2.<br>
<br>
Back in March I ran performance comparisons for SQL/postgres and Titan (0.5.4)<br>
on Tinkerpop2 and concluded that Titan was the best choice then.<br>
<br>
After completing a simplistic port of inventory to Tinkerpop3 (not taking<br>
advantage of any new features or opportunities to simplify inventory<br>
codebase), I've run the performance tests again for the 2 new backends - Titan<br>
1.0 and Sqlg (on postgres).<br>
<br>
This time the results are not so clear as the last time.<br>
>From the charts [1] you can see that Postgres is actually quite a bit faster<br>
on reads and can better handle concurrent read access while Titan shines in<br>
writes (arguably thanks to Cassandra as its storage).<br>
<br>
Of course, I can imagine that the read performance advantage of Postgres would<br>
decrease with the growing amount of data stored (the tests ran with the<br>
inventory size of ~10k entities) but I am quite positive we'd get competitive<br>
read performance from both solutions up to the sizes of inventory we<br>
anticipate (100k-1M entities).<br>
<br>
Now the question is whether the insert performance is something we should be<br>
worried about in Postgres too much. IMHO, there should be some room for<br>
improvement in Sqlg and also our move to /sync for agent synchronization would<br>
make this less of a problem (because there would be not that many initial<br>
imports that would create vast amounts of entities).<br>
<br>
Nevertheless I currently cannot say who is the "winner" here. Each backend has<br>
its pros and cons:<br>
<br>
Titan:<br>
Pros:<br>
- high write throughput<br>
- backed by cassandra<br>
<br>
Cons:<br>
- slower reads<br>
- project virtually dead<br>
- complex codebase (self-made fixes unlikely)<br>
<br>
Sqlg:<br>
Pros:<br>
- small codebase<br>
- everybody knows SQL<br>
- faster reads<br>
- faster concurrent reads<br>
<br>
Cons:<br>
- slow writes<br>
- another backend needed (Postgres)<br>
<br>
Therefore my intention here is to go forward with a "proper" port to<br>
Tinkerpop3 with Titan still enabled but focus primarily on Sqlg to see if we<br>
can do anything with the write performance.<br>
<br>
IMHO, any choice we make is "workable" as it is even today but we need to<br>
weigh in the productization requirements. For those Sqlg with its small dep<br>
footprint and postgres backend seems preferable to the huge dependency mess of<br>
Titan.<br>
<br>
[1] <a href="https://dashboards.ly/ua-TtqrpCXcQ3fnjezP5phKhc" rel="noreferrer" target="_blank">https://dashboards.ly/ua-<wbr>TtqrpCXcQ3fnjezP5phKhc</a><br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Lukas Krejci<br>
______________________________<wbr>_________________<br>
hawkular-dev mailing list<br>
<a href="mailto:hawkular-dev@lists.jboss.org">hawkular-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/hawkular-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/hawkular-dev</a><br>
<br>
<br>
</font></span></blockquote></div><br></div>