[infinispan-dev] Re: Infinispan and search APIs

[Manik Surtani]

On 19 May 2009, at 01:44, Ray Hilton wrote:

> Hi guys,
>
> I've been trying to get jbc-searchable working with JbossCache 3,  
> but then I noticed this thread for Infinispan.  I am trying to build  
> a proof-of-concept for a distributed searchable cache, and obviously  
> infinispan with hibernate search would be an ideal solution.  At  
> what stage of development is this?  Is it possible to use the  
> existing jbc-searchable code with infinispan?  Should I perhaps  
> stick with jbc for now?

This isn't in Infinispan as yet.  The plan is to add querying support,  
but it won't make our next major release.  Unless you feel like  
contributing it.  :-)

> On a related note, I like the idea that each node builds and stores  
> its own indices for data it currently hosts.  It would also be nice  
> if these indices could be further partitioned (in our case, we would  
> like to do this per-day, other situations maybe based on a modulous  
> of the hash code) to reduce the size of the indices.  Replicated  
> indices are ok if they dont change that often, but ours will be  
> updated almost constantly, and so you either need to nominate one  
> node to do the work, or have some distributed locking in place,  
> which would probably kill performance.  Building an index locally  
> also provides protection against index corruption by having  
> redundant copies around the grid.  We are intending to do  
> deduplication and score normalization ourselves.
>
> I also tried using the jbosscache-lucene Directory implementation  
> that Manik wrote, but things started to die when lucene tried to  
> merge segments.  Has there been any work to create a infinispan- 
> backed lucene Directory?

There hasn't been any effort as yet, and with Vladimir's recent work  
on the lock() API, I expect Infinispan to be a far better fit for a  
Lucene directory than JBoss Cache is.  So again, if you are interested  
in contributing this, perhaps based on my JBoss Cache implementation,  
it would be much appreciated.

Cheers
Manik

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org