[infinispan-dev] Some memory usage data points of the Hibernate Search / Lucene query engine.

[Sanne Grinovero]

Hi all,
at our last meeting we had some chats about heap usage at runtime.

I wasn't actually investigating this, but since I was running some
tests now I thought it was interesting to share the following figures
about the Hibernate Search / Lucene query engine.

# Index size
My test is running with an off-heap index of about 5 million entries,
which translates to about 170MB of disk space.

# Query type
We're running a large query which matches every single one of the 5
million entries, and sorting the results on a Float property.

# TLAB usage
Running such a query takes 2.78 MB of TLAB; this data is never
promoted so it's "cheap" to collect as there's no much interaction
with other threads.

# Beyond TLAB
Allocating zero bytes ;)

In terms of memory usage this looks quite good; kudos to the Lucene
team of course as they do the heavy lifting, but I'm proud of the
Hibernate Search team as well as we don't add significant overhead.
We still have some work to do, as while memory looks good the sorting
on Floats is still very heavy.. but we'll figure it out.

Looking forward to see similar metrics from get/put tests on Infinispan ;-)

N.B. this is an "off heap" index, but even if I use an "on heap"
index, above figures are pretty much the same, and the performance
doesn't get higher, but slower.

Thanks,
Sanne