[infinispan-dev] ISPN 200

[Israel Lacerra]

Hi Sanne,

Thank you for the feedback! I will see this ParallelMultiSearcher today and
all you other comments.

I have a little doubt, about the development environment that you use. I'm
using Eclipse, but since the migration to Git, I'm having some troubles. The
Git plugin is not so good...

Are you using Eclipse or another IDE? If you using Eclipse, are you using
another GUI (outside Eclipse) to synchronize and see all changes?

thanks!

Israel

On Mon, Jan 10, 2011 at 10:53 AM, Sanne Grinovero <sanne.grinovero at gmail.com
> wrote:

> Hi Israel,
> thank you very much, I've been very eager to read some code on this.
>
> I've started to add some very minor comments to github; it would
> greatly help if you could add some
> heading javadoc to each class you created, for example what's the
> purpose and role of QueryBox?
>
> About sorting, it's good to have a starter, but I don't think we can
> use reflection so massively, nor
> rely on Comparator.
>
> Sort fields aren't mandated to be stored, so it's not an option to
> pre-extract them from the index either;
> bottom line, it's not enough to retrieve only the matching objects
> from the remote node, you have to fetch
> also some index lowlevel metadata to be able to properly merge the
> results using the appropriate sorting.
>
> I just found this code which might be very useful as inspiration, it's
> designed to merge results
> from searches performed in different threads, but if you re-think the
> executor as remote node executors,
> you should be set:
> org.apache.lucene.search.ParallelMultiSearcher.search(Weight, Filter, int,
> Sort)
>
> So you have each node perform a Query on his local index, but then the
> final sort is performed on the requesting
> node again processing the same TopFieldDocs which each remote node created.
> When returning the matching value objects, return the TopFieldDocs
> too. Actually I'd say it should be fine to retrieve
> the values at a later (second phase) step, as it's quite possible
> people want to retrieve only the "top 100" or similar queries.
>
> Finally, it would be great to see some test. We don't need extreme
> code coverage at this point, but it would greatly help to
> have at least a two nodes scenario with a simple sort being performed,
> it's also nice for people
> wanting to look into it, as an example of how to use your code and
> what are the entry points.
>
> thank you, great job.
> Feel free to ask anything, by email or on IRC.
>
> Sanne
>
>
> 2011/1/7 Israel Lacerra <israeldl at gmail.com>:
> > I've pushed some code in https://github.com/israeldl/infinispan/
> >
> > This is not the final version. Is just to you take a look and tell me
> what
> > you think. This commit have some problems (I'm fighting with Git yet).
> > Ignore the changes in demos/*.
> >
> > To use the distributed query, we have to call
> > QueryFactory.getDistributedQuery.
> >
> > There must be a lot things that I have to improve in the code, but I hope
> it
> > is not in a totally wrong way!
> >
> > Some thoughts:
> > - In the QueryBox, I made a simple eviction policy, and probably not the
> > best. I maintain up to 300 lazyIterators. What you think is a good
> policy??
> > - To make the sort, I had to force the fields used in that sort to
> implement
> > Comparable. I could not use comparator methods behind the Sort class of
> > Lucene. One option would be to create another way to user create the sort
> > method and then from there we would create the lucene Sort. What you
> think?
> > - I created a ClusteredQueryFacade on org.infinispan.commands, because I
> > initialize this command in
> CommandsFactoryImpl.initializeReplicableCommand.
> > There is a better way??
> > - The sort is made in a naive way. If we have too many nodes, probably we
> > will have problems. Should I improve this? In a real case, the number of
> > nodes will be too big?
> >
> >
> > thanks!
> >
> > Israel
> >
> > On Tue, Jan 4, 2011 at 2:49 PM, Manik Surtani <manik at jboss.org> wrote:
> >>
> >> On 30 Dec 2010, at 18:34, Israel Lacerra wrote:
> >>
> >> > Manik,
> >> >
> >> > I'm gonna need a few more days. I'll probably send you the code on the
> >> > middle of the next week (about jan/6).
> >> >
> >> > Guys,
> >> >
> >> > About the way I'll send the code, I think the best is to make a fork
> on
> >> > github and then I push my changes on this fork and send you the url of
> the
> >> > fork. Right?
> >>
> >> Yes.  Pls post the link to this mail list so others can take a look as
> >> well.
> >>
> >> Cheers
> >> Manik
> >>
> >> --
> >> Manik Surtani
> >> manik at jboss.org
> >> twitter.com/maniksurtani
> >>
> >> Lead, Infinispan
> >> http://www.infinispan.org
> >>
> >>
> >>
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20110113/d5b36471/attachment.html