[infinispan-dev] Re: Infinispan and search APIs

[Manik Surtani]

On 21 May 2009, at 00:15, Ray Hilton wrote:

> On 19/05/2009, at 7:56 PM, Manik Surtani wrote:
>
>>> I've been trying to get jbc-searchable working with JbossCache 3,  
>>> but then I noticed this thread for Infinispan.  I am trying to  
>>> build a proof-of-concept for a distributed searchable cache, and  
>>> obviously infinispan with hibernate search would be an ideal  
>>> solution.  At what stage of development is this?  Is it possible  
>>> to use the existing jbc-searchable code with infinispan?  Should I  
>>> perhaps stick with jbc for now?
>>
>> This isn't in Infinispan as yet.  The plan is to add querying  
>> support, but it won't make our next major release.  Unless you feel  
>> like contributing it.  :-)
>
> I'd be happy to take a look.  What would be the preferred to monitor  
> what data is being stored on the local node?  I took a look at the  
> interceptors, but Im unsure as to where to start.  In theory, I  
> guess you just need to know what data is being stored or evicted  
> from the local node and update an index accordingly.

I would expect we do this in an interceptor.  The interceptor adds or  
updates indexes in Hibernate Search accordingly.

We'd need a new API to deal with queries, you should take a look at  
JBoss Cache - Searchable [1] and use it as a starting point.  JBCS  
does *not* use interceptors though, instead it attaches a listener to  
JBC.  I jotted some notes in improving this to use interceptors [2].   
I believe the approach for Infinispan should use interceptors.

Cheers
Manik

[1] http://anonsvn.jboss.org/repos/jbosscache/searchable/trunk/
[2] http://www.jboss.org/community/wiki/JBossCache-SearchableEdition-Designs


>
>>> I also tried using the jbosscache-lucene Directory implementation  
>>> that Manik wrote, but things started to die when lucene tried to  
>>> merge segments.  Has there been any work to create a infinispan- 
>>> backed lucene Directory?
>>
>> There hasn't been any effort as yet, and with Vladimir's recent  
>> work on the lock() API, I expect Infinispan to be a far better fit  
>> for a Lucene directory than JBoss Cache is.  So again, if you are  
>> interested in contributing this, perhaps based on my JBoss Cache  
>> implementation, it would be much appreciated.
> I'm leaning away from storing the Directory in-grid at the moment,  
> feels like a more complex and error-prone solution than keeping  
> indices on each machine (no distributed locks, etc).  Redundant  
> indices also protects against index corruption, something we seem to  
> get a lot of in our current implementation (using ruby's ferret,  
> although I am sure Lucene is more reliable).
>
> Ray

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org