Hi Sanne,<br><br>Thank you for the feedback! I will see this ParallelMultiSearcher today and all you other comments.<br><br>I have a little doubt, about the development environment that you use. I'm using Eclipse, but since the migration to Git, I'm having some troubles. The Git plugin is not so good... <br>
<br>Are you using Eclipse or another IDE? If you using Eclipse, are you using another GUI (outside Eclipse) to synchronize and see all changes?<br><br>thanks!<br><br>Israel<br><br><div class="gmail_quote">On Mon, Jan 10, 2011 at 10:53 AM, Sanne Grinovero <span dir="ltr"><<a href="mailto:sanne.grinovero@gmail.com">sanne.grinovero@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi Israel,<br>
thank you very much, I've been very eager to read some code on this.<br>
<br>
I've started to add some very minor comments to github; it would<br>
greatly help if you could add some<br>
heading javadoc to each class you created, for example what's the<br>
purpose and role of QueryBox?<br>
<br>
About sorting, it's good to have a starter, but I don't think we can<br>
use reflection so massively, nor<br>
rely on Comparator.<br>
<br>
Sort fields aren't mandated to be stored, so it's not an option to<br>
pre-extract them from the index either;<br>
bottom line, it's not enough to retrieve only the matching objects<br>
from the remote node, you have to fetch<br>
also some index lowlevel metadata to be able to properly merge the<br>
results using the appropriate sorting.<br>
<br>
I just found this code which might be very useful as inspiration, it's<br>
designed to merge results<br>
from searches performed in different threads, but if you re-think the<br>
executor as remote node executors,<br>
you should be set:<br>
org.apache.lucene.search.ParallelMultiSearcher.search(Weight, Filter, int, Sort)<br>
<br>
So you have each node perform a Query on his local index, but then the<br>
final sort is performed on the requesting<br>
node again processing the same TopFieldDocs which each remote node created.<br>
When returning the matching value objects, return the TopFieldDocs<br>
too. Actually I'd say it should be fine to retrieve<br>
the values at a later (second phase) step, as it's quite possible<br>
people want to retrieve only the "top 100" or similar queries.<br>
<br>
Finally, it would be great to see some test. We don't need extreme<br>
code coverage at this point, but it would greatly help to<br>
have at least a two nodes scenario with a simple sort being performed,<br>
it's also nice for people<br>
wanting to look into it, as an example of how to use your code and<br>
what are the entry points.<br>
<br>
thank you, great job.<br>
Feel free to ask anything, by email or on IRC.<br>
<br>
Sanne<br>
<br>
<br>
2011/1/7 Israel Lacerra <<a href="mailto:israeldl@gmail.com">israeldl@gmail.com</a>>:<br>
<div><div></div><div class="h5">> I've pushed some code in <a href="https://github.com/israeldl/infinispan/" target="_blank">https://github.com/israeldl/infinispan/</a><br>
><br>
> This is not the final version. Is just to you take a look and tell me what<br>
> you think. This commit have some problems (I'm fighting with Git yet).<br>
> Ignore the changes in demos/*.<br>
><br>
> To use the distributed query, we have to call<br>
> QueryFactory.getDistributedQuery.<br>
><br>
> There must be a lot things that I have to improve in the code, but I hope it<br>
> is not in a totally wrong way!<br>
><br>
> Some thoughts:<br>
> - In the QueryBox, I made a simple eviction policy, and probably not the<br>
> best. I maintain up to 300 lazyIterators. What you think is a good policy??<br>
> - To make the sort, I had to force the fields used in that sort to implement<br>
> Comparable. I could not use comparator methods behind the Sort class of<br>
> Lucene. One option would be to create another way to user create the sort<br>
> method and then from there we would create the lucene Sort. What you think?<br>
> - I created a ClusteredQueryFacade on org.infinispan.commands, because I<br>
> initialize this command in CommandsFactoryImpl.initializeReplicableCommand.<br>
> There is a better way??<br>
> - The sort is made in a naive way. If we have too many nodes, probably we<br>
> will have problems. Should I improve this? In a real case, the number of<br>
> nodes will be too big?<br>
><br>
><br>
> thanks!<br>
><br>
> Israel<br>
><br>
> On Tue, Jan 4, 2011 at 2:49 PM, Manik Surtani <<a href="mailto:manik@jboss.org">manik@jboss.org</a>> wrote:<br>
>><br>
>> On 30 Dec 2010, at 18:34, Israel Lacerra wrote:<br>
>><br>
>> > Manik,<br>
>> ><br>
>> > I'm gonna need a few more days. I'll probably send you the code on the<br>
>> > middle of the next week (about jan/6).<br>
>> ><br>
>> > Guys,<br>
>> ><br>
>> > About the way I'll send the code, I think the best is to make a fork on<br>
>> > github and then I push my changes on this fork and send you the url of the<br>
>> > fork. Right?<br>
>><br>
>> Yes. Pls post the link to this mail list so others can take a look as<br>
>> well.<br>
>><br>
>> Cheers<br>
>> Manik<br>
>><br>
>> --<br>
>> Manik Surtani<br>
>> <a href="mailto:manik@jboss.org">manik@jboss.org</a><br>
>> <a href="http://twitter.com/maniksurtani" target="_blank">twitter.com/maniksurtani</a><br>
>><br>
>> Lead, Infinispan<br>
>> <a href="http://www.infinispan.org" target="_blank">http://www.infinispan.org</a><br>
>><br>
>><br>
>><br>
><br>
><br>
</div></div><div><div></div><div class="h5">> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
><br>
<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></div></div></blockquote></div><br>