[infinispan-dev] asynchronous change manager for high load Lucene

[Emmanuel Bernard]

On 29 mars 2010, at 14:04, Manik Surtani wrote:

>> 
>> The Lucene Directory is doing well in read-most situations, and a nice
>> to have for huge sized indexes to easily sync them in dynamic-topology
>> clusters.
>> But when the many participant nodes all potentially apply changes, a
>> global index lock needs to be acquired; this is safely achievable by
>> using the LockFactory implementations provided in the lucene-directory
>> package but of course doesn't scale.
> 
> This is because each time an update is made, all parts of the index need to be updated?  Can this not be done in a finer-grained manner where nodes only lock and update the chunks they need to update?  Caveat: I have no idea of Lucene index formats, so this may make no sense whatsoever.  :)

Nope. The finest granularity is the Lucene index. We do split the indexed data in multiple indexes to reduce the granularity but at the index level, you need one exclusive lock.