I don't know the stability of snapshot but if it works well we
certainly should move there as it releases us from the mostly useless
lock code. Is snapshot Directory implementation agnostic? Or does it
rely on FS?
On Dec 16, 2008, at 11:18, Sanne Grinovero wrote:
2008/12/16 Hardy Ferentschik <hibernate(a)ferentschik.de>:
> Hi Sanne,
> On Mon, 15 Dec 2008 20:15:52 +0100, Sanne Grinovero
> <sanne.grinovero(a)gmail.com> wrote:
>> some more steps towards mass indexing
> What exactly is actually your definition of "mass indexing"? The
> use of
> to (re-)build your index?
I have split the problem it two:
first part is to just speedup the whole process changing
the backend only, so people will get the new benefits using the
described in the book and in the reference docs: so yes I mean using
The second part is to add a new API which wraps the complexity of
the object loading, and uses
FullTextSession.index() under the covers (I hope to). It is not really
clear at the moment
which arguments this new API will need, as people may want to finetune
the process in several ways,
but it will look like the same as the recommended approach, but using
>> do you agree I'll drop the capability to use an IndexReader to make
>> changes to the index?
>> This implies I'll simplify the backend by removing all methods
>> on an IndexReader (they are not needed anymore),
>> and is required to reuse the IndexWriter as next improvement.
> I assume you are talking about removing performWork(LuceneWork work,
> IndexReader reader)
> from the LuceneWorkDelegate interface. I think that makes sense. I
> think we
> still agree
> that it is a good idea to only use the IndexWriter to apply changes.
Nice I'll start with this as it really helps minimizing the complexity
of the backend.
>> Also we'll need to change the FSMasterDirectoryProvider to use
>> Lucene's snapshotting feature instead of the DirectoryProvider
>> lock (is
>> Another solution would be to replace the lock-holding with a signal
>> like "please release it to me for a moment"
>> or even delegate the task to the thread owning the IndexWriter.
> Not sure about this one. I haven't looked into this snapshot
> feature yet.
This feature makes it possible
to make consistent copies of the index even when an IndexWriter is
being used to make changes.
I would need this to make sure the index can still be hot copied even
when we hold a lock
for longer times (avoid closing the IndexWriter).
>> BTW this will mean the DirectoryProvider won't be used by anybody.
Sorry, of course not, I meant to say the "DirectoryProvider-specific
It is currently used only to achieve mutual exclusion between the
IndexWriter activity and the
FSMasterDirectoryProvider's copy task, so if we implement the
it won't be useful anymore.