Re: [hibernate-dev] Hibernate Search: FieldCache usage for classtype extraction from Documents
On Wed, 02 Mar 2011 19:48:33 +0100, Sanne Grinovero <sanne(a)hibernate.org>
wrote
so it's quite possible that this approach takes hostage of a lot
of
memory.
I guess that was our fear from the beginning.
So to cope with this, I've now built a custom fieldCache and a
specialized Collector, which extract the data and keep it in memory
not as String but as Class instances, which have the advantage of
pointing to the same instances.
I will have a look at it tomorrow.
profiling, it's all about an innocently looking Map.put() which
I'm
now using in the custom collector.
Basically, I'm going to need to change the approach, or kill the
feature. I'm now collecting values into an array and will see how it
goes.
In case someone wants to have a look, there's a working branch here:
https://github.com/Sanne/hibernate-search/tree/HSEARCH-531
and this is the line which is taking now 93% CPU time during a fulltext
search:
https://github.com/Sanne/hibernate-search/commit/735c4644834cc516effa33f6...
(Hardy beware of your map usage! this is worse than we expected.. you
definitely have to switch to a per-field collector instance if
possible)
Right. Going to the chained Collector approach is next on my todo list.
Maybe we can have a chat tomorrow about this.
--Hardy