[hibernate-dev] RE: hibernate-dev Digest, Vol 12, Issue 10

[Hardy Ferentschik]

Hi,

> These three settings, mergeFactor, maxMergeDocs and minMergeDocs, are
> critical to scalability as the number of records to index becomes very
> large. Currently I work with tables containing millions of records and  
> the
> ability to adjust these values to balance number of index files vs.  
> memory
> usage vs. disk access is vital. I suggest these be exposed to the user.
>
> One thing that should be considered is that for maximum benefit these  
> values
> should be adjustable. During complete index builds from scratch they  
> should
> contain one set of values. During normal use they should contain another.
> This maximizes their potential.

What exactly do you have in mind? Some sort of Lucene Management API which  
allows
the user to programmatically change these values? Or JMX? Or just two sets  
of
properties, one for full reindexing and one for 'normal' use?
I somehow like the idea of a management API. Via this API you could set  
the indexing
parameters, but also trigger eg full index rebuilds or index optimizations.
JMX would be nice as well. It would allow to change parameters on the fly
and allow easy fine-tuning.

> I also suggest that accurate and detailed documentation be included on
> these. As soon as I get out from under the load I have at work I'll try  
> to
> hep.

Agreed. Documentation in general needs some work. I found it very hard to  
find some information
about Hibernate projection. It had to work my way backwards from a  
projection testcase in HSearch.
I think HSearch with projections is a great step forward, but we need some  
code examples so
that people can see how to use them. I will try to add something to the  
wiki.


--Hardy