[infinispan-dev] [hibernate-dev] Distributed queries

Sun Sep 20 00:11:26 EDT 2009

Sorry, just saw Sanne's email regarding the two approaches, should
have read the whole thread more thoroughly before posting :P

So regarding point 1) in that email, would that be similar to the
second point in my email?  where a (or each) node would watch objects
being committed and evicted and update the index accordingly
(disregarding where the index is actually stored... thinking about it,
that is just a matter of Directory implementation, so could be local
or in-grid).

Navin, could you point me in the direction of where that code lives at
the moment?  Again, I'd be happy to help out in some way.

Ray

On Sun, Sep 20, 2009 at 2:01 PM, Ray Hilton <ray at wirestorm.net> wrote:
> Hi guys,
>
> I've been following the distributed query stuff with interest, but
> this is the first time I'm posting, so please excuse the lack of
> intimate knowledge of Infinispan.  Basically, I have been working on a
> project that could really do with the Holy Grail of a distributed
> query-able cache and I really liked the look of using JBossCache + the
> Lucene Directory implementation that Manik wrote a while back.  I then
> noticed Infinispan and talk of building querying directly into the
> project and figured that it would be worthwhile waiting to see how
> that panned out.
>
> I've thought a bit about how something like this might work, I'm not
> sure if this will be in any way helpful, but here goes:  I guess there
> are two approaches:  1) store the index (or partitioned indices) in
> the grid and sync it to a node to do a particular query or 2) each
> node has an index for the data it currently caches.  We preferred the
> second idea as it offers a natural way to partition the indices (i.e.
> however infinispan is configured to do it).  The first option would
> mean you end up with either a monolithic index in the grid, or
> partitions based on, say, date, that have to be sync'd en-mass to
> whichever node(s) are doing a query.  I realise that the second
> technique would produce duplicates, but Im sure there would be a way
> to eliminate dupes based on the object's uuid (something im pretty
> sure infinispan already has a notion of).
>
> We would also need to come up with a way or normalising the scoring
> across all partitions (regardless of which method is used).  I have
> seen this done before, and it would basically involve, per-query,
> finding out the term frequency of the various keywords across the
> entire index, or at least enough of it to produce a representative
> value.  This would be used to calculate the score for each hit when
> doing the actual search, and thus the ranking.
>
> We have had issues with index corruption in the past as well (probably
> due to programming bugs rather than lucene).  Making each node
> responsible for its own index will make it very easy to throw corrupt
> indices away and re-generate new ones.
>
> I did take a look at the visitor stuff in Infinispan before, but I
> wasn't really sure where the best place to hook into would be to find
> out which objects are being stored locally or evicted.  If someone has
> a good idea of where to start, I'd be happy to lend a hand to to this
> effort!
>
> Ray
>
>
> On Sat, Sep 19, 2009 at 8:43 PM, Michael Neale <michael.neale at gmail.com> wrote:
>> I think you just stuck a pin in the bubble that normally says "magic
>> happens here" ;)
>>
>> How much of this did you tackle regarding hibernate search that could
>> be applied here?
>>
>> (you final point re duplication may have some "flexibility" I think ?)
>>
>> On Fri, Sep 18, 2009 at 6:18 PM, Emmanuel Bernard
>> <emmanuel at hibernate.org> wrote:
>>> Neither 1 nor 2 imply *distributed* queries.
>>>
>>> The hard parts with distributed queries (ie executed on a grid and
>>> recomposed) are:
>>>  - making sure you ask all the nodes where the index is distributed
>>> (you can't miss a node)
>>>  - find a way to index only a subset of the data in a given index (on
>>> a given node). Applying the Infinispan distribution routine to the
>>> InfinispanDirectory does not do that, it chunks data arbitrarily.
>>>  - be able to rebuild a given index on a givne node (ie remember
>>> which element were indexed)
>>>  - you need to find a way to distribute your data without
>>> duplication. If a key is indexed multiple times, then you end up with
>>> duplicated results that can't trivially be de-duplicated.
>>>
>>> Happy thinking.
>>>
>>> On 17 sept. 09, at 10:32, Sanne Grinovero wrote:
>>>
>>>> 2009/9/17 Michael Neale <michael.neale at gmail.com>:
>>>>> I am still not entirely sure what I am asking, but look forward for
>>>>> your merged in changes (they are in another branch right now yes?).
>>>>>
>>>>> Yes I mean querying objects - I was under the impression that lucene
>>>>> was used for the indexing of the data to service these queries?
>>>>
>>>> Sure, to clarify: there's work going on on two different aspects,
>>>> which
>>>> complement each other in the ideal setup:
>>>>
>>>> 1) Be able to query a Lucene index (wherever you store that) to find
>>>> objects
>>>> which are located inside Infinispan; this is about how to search
>>>> them and how
>>>> to maintain the index in synch with Infinispan's content.
>>>>
>>>> 2) Store a Lucene index inside Infinispan, instead of, for example,
>>>> filesystem.
>>>> In this case we're not concerned about what you index, the Lucene
>>>> interface
>>>> is the usual one and you should be able to replace the Directory
>>>> implementation in existing applications.
>>>>
>>>> So 1) is the branch you've found, and Navin is working on that, 2)
>>>> is not yet
>>>> in subversion, the latest patch is attached to other thread by
>>>> Łukasz,
>>>> and is to be applied
>>>> on Hibernate Search's trunk (and depends on Infinispan).
>>>>
>>>>>
>>>>> On Wed, Sep 16, 2009 at 10:32 PM, Navin Surtani
>>>>> <nsurtani at redhat.com> wrote:
>>>>>>
>>>>>> On 16 Sep 2009, at 12:25, Michael Neale wrote:
>>>>>>
>>>>>>> oh ok nice - could you point me at which branch to try to find some
>>>>>>> tests to play with?
>>>>>>
>>>>>> If you're talking about Querying objects in Infinispan: -
>>>>>>
>>>>>> The eventual goal is to be able to have different configurations on
>>>>>> how you want to index your data. Manik has given me the 'OK' to
>>>>>> push a
>>>>>> simple query interface for CR1 for Monday/Tuesday.
>>>>>>
>>>>>> I'm kind-of pressed with getting the code working for this and also
>>>>>> between moving house and lack of internet there I'll be a bit quiet.
>>>>>> However, I'll get a wiki up by the end of the week about how this
>>>>>> all
>>>>>> works.
>>>>>>
>>>>>> However if you're not then I assume you're talking about using
>>>>>> Lucene
>>>>>> to index into Infinispan?
>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 16, 2009 at 6:05 PM, Sanne Grinovero
>>>>>>> <sanne.grinovero at gmail.com> wrote:
>>>>>>>> 2009/9/16 Michael Neale <michael.neale at gmail.com>:
>>>>>>>>> regarding indexing and queries - is the current aim to not
>>>>>>>>> require
>>>>>>>>> that the index for the entire data grid exist on a single node?
>>>>>>>>>
>>>>>>>>> (asking as a potential user who is wrestling with lucene
>>>>>>>>> indexes at
>>>>>>>>> the moment is curious).
>>>>>>>>
>>>>>>>> Yes the concept is to store the Lucene index itself in the grid,
>>>>>>>> so
>>>>>>>> it will
>>>>>>>> be distributed, and the segments you use most get cached locally.
>>>>>>>> At the moment you have to select only one node to write to the
>>>>>>>> index,
>>>>>>>> but all other nodes should be able to read.
>>>>>>>> Feel free to test it as we are needing feedback.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Michael D Neale
>>>>>>>>> home: www.michaelneale.net
>>>>>>>>> blog: michaelneale.blogspot.com
>>>>>>>>> _______________________________________________
>>>>>>>>> infinispan-dev mailing list
>>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Michael D Neale
>>>>>>> home: www.michaelneale.net
>>>>>>> blog: michaelneale.blogspot.com
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>> Navin Surtani
>>>>>>
>>>>>> Intern Infinispan
>>>>>> Intern JBoss Cache Searchable
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Michael D Neale
>>>>> home: www.michaelneale.net
>>>>> blog: michaelneale.blogspot.com
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>
>>>> _______________________________________________
>>>> hibernate-dev mailing list
>>>> hibernate-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> --
>> Michael D Neale
>> home: www.michaelneale.net
>> blog: michaelneale.blogspot.com
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> --
> Ray Hilton
> -
>         email: ray at wirestorm.net
>  melbourne: +61 (0) 3 9077 0513
>       mobile: +61 (0) 430 484 708
>

-- 
Ray Hilton
-
         email: ray at wirestorm.net
 melbourne: +61 (0) 3 9077 0513
       mobile: +61 (0) 430 484 708