[hibernate-dev] Search: dropping support for changes through IndexReader

[Emmanuel Bernard]

I don't know the stability of snapshot but if it works well we  
certainly should move there as it releases us from the mostly useless  
lock code. Is snapshot Directory implementation agnostic? Or does it  
rely on FS?

On  Dec 16, 2008, at 11:18, Sanne Grinovero wrote:

> 2008/12/16 Hardy Ferentschik <hibernate at ferentschik.de>:
>> Hi Sanne,
>>
>> On Mon, 15 Dec 2008 20:15:52 +0100, Sanne Grinovero
>> <sanne.grinovero at gmail.com> wrote:
>>
>>> some more steps towards mass indexing
>>
>> What exactly is actually your definition of "mass indexing"? The  
>> use of
>> FullTextSession.index()
>> to (re-)build your index?
>
> I have split the problem it two:
>   first part is to just speedup the whole process changing
> the backend only, so people will get the new benefits using the
> recommended procedure
> described in the book and in the reference docs: so yes I mean using
> "FullTextSession.index()".
>
>   The second part is to add a new API which wraps the complexity of
> the object loading, and uses
> FullTextSession.index() under the covers (I hope to). It is not really
> clear at the moment
> which arguments this new API will need, as people may want to finetune
> the process in several ways,
> but it will look like the same as the recommended approach, but using
> several threads.
>
>>
>>> do you agree I'll drop the capability to use an IndexReader to make
>>> changes to the index?
>>> This implies I'll simplify the backend by removing all methods  
>>> working
>>> on an IndexReader (they are not needed anymore),
>>> and is required to reuse the IndexWriter as next improvement.
>>
>> I assume you are talking about removing performWork(LuceneWork work,
>> IndexReader reader)
>> from the LuceneWorkDelegate interface. I think that makes sense. I  
>> think we
>> still agree
>> that it is a good idea to only use the IndexWriter to apply changes.
>
> Nice I'll start with this as it really helps minimizing the complexity
> of the backend.
>
>>
>>> Also we'll need to change the FSMasterDirectoryProvider to use
>>> Lucene's snapshotting feature instead of the DirectoryProvider  
>>> lock (is
>>> HSEARCH-152).
>>> Another solution would be to replace the lock-holding with a signal
>>> like "please release it to me for a moment"
>>> or even delegate the task to the thread owning the IndexWriter.
>>
>> Not sure about this one. I haven't looked into this snapshot  
>> feature yet.
>
> This feature makes it possible
> to make consistent copies of the index even when an IndexWriter is
> being used to make changes.
> I would need this to make sure the index can still be hot copied even
> when we hold a lock
> for longer times (avoid closing the IndexWriter).
>
>>
>>> BTW this will mean the DirectoryProvider won't be used by anybody.
>> Really?
> Sorry, of course not, I meant to say the "DirectoryProvider-specific  
> LOCK";
> It is currently used only to achieve mutual exclusion between the
> IndexWriter activity and the
> FSMasterDirectoryProvider's copy task, so if we implement the  
> snapshotting
> it won't be useful anymore.
>
>>
>> --Hardy
>>
>
> Sanne