<div>Hi,</div><div><br></div><div>#2. I am using Intellij 8.1. I have downloaded from wiki xml file with codestyle for intellij, but it's still </div><div>a little bit different from currently existing in HS (i.e. imports order or line length).</div>
<div><br></div><div>#4. The "listenerClassName" from JGroupsBackendQueueProcessorFactory </div><div>is mandatory but only in case it is master node, then it specifies class responsible for receiving messages from </div>
<div>slave's. I assumed if that option exist it is master node, otherwise slave. Falling back to a Lucene implementation </div><div>was intended to avoid problem from JMS backend; setting up master node to act also like slave node led to message </div>
<div>duplication - MDB received message, recreate it and put it back to the queue. JGroupsBackendQueueProcessorFactory </div><div>for master node create also LuceneFactory, and received from slaves Lucene works are processed by LuceneProcessor - message duplication is avoided. It is the only one reason why I used that. </div>
<div>User is supposed to create readers and receivers with this same - JGroupsBackendQueueProcessorFactory.</div><div><br></div><div>//receiver sample configuration</div><div><property name="hibernate.search.worker.backend" value="jgroups"/> </div>
<div><property name="hibernate.search.worker.jgroups.channel_name" value="hs_jg_channel"/></div><div><property name="hibernate.search.worker.jgroups.listener_class" </div><div>value="pl.lmoren.master.JGroupsMessageReceiverImpl"/></div>
<div><br></div><div><br></div><div>//producers sample configuraion</div><div><property name="hibernate.search.worker.backend" value="jgroups"/></div><div><property name="hibernate.search.worker.jgroups.channel_name" value="hs_jg_channel"/></div>
<div><br></div><div>However, in receiver's case JGroups Factory create Lucene Processors in the context of the problem described above. </div><div>JGroups Factory for master node just initialize communication channel, but later all work is done by Lucene Backend, so this same like in JMS.</div>
<div>That solution is not fully consistent with JMS backend one, it was forced by I think different nature of JMS and </div><div>JGroups.</div><div>In JMS receiver configuration was done in app server config file, in JGroups I placed it in hibernate configuration.</div>
<div>What do you think about such design?</div><div><br></div><div><br></div><div>#5.That is question, I was also thinking about, to make clustering more transparent. </div><div>Hovever I haven't found good idea for that. The purpose of extending JGroupsAbstractMessageReceiver's by user was to implement getSession method, where session used by backend could be created. I suppose that without this method I would not know if Hibernate sessionFactory should come from i.e. persistence context, looked up from JNDI or some helper class.</div>
<div><br></div><div>#6. Problem with JGroups exists if it tries to multicast traffic towards the ISP, not internal network, then I think packages are dropped. I will look through that in the weekend.</div><div><br></div><div>
<br></div><div>Lukasz</div><div><br></div><br><div class="gmail_quote">2009/6/10 Sanne Grinovero <span dir="ltr"><<a href="mailto:sanne.grinovero@gmail.com">sanne.grinovero@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi Lukasz,<br>
I've been looking into your code; I have some comments but please<br>
forgive me as I don't have any real experience about JGroups, so I'll<br>
only tell you how much I see this code fit into Hibernate Search.<br>
<br>
1) The maven dependency upon JGroups should probably be of type<br>
"optional", so please make sure search is also going to work fine for<br>
people which are not really interested in this work and having the<br>
jgroups.jar around, even if others will love it.<br>
<br>
2) Look out for style, especially white spacing; for example in<br>
BatchedQueueingProcessor look at the formatting difference of line 91<br>
compared to 82,85 or 88.<br>
Which IDE are you using? we have some template settings ready to help<br>
with this, you can find them on the <a href="http://hibernate.org" target="_blank">hibernate.org</a> wiki's.<br>
<br>
3) JGroupsBackendQueueProcessor is a Runnable setting two arguments in<br>
the constructor, they should be final to make sure they are set before<br>
they are run in another thread;<br>
just add "final" modifier to lines 19 and 20.<br>
<br>
4)JGroupsBackendQueueProcessorFactory's design:<br>
It looks like a "listenerClassName" is mandatory and not providing a<br>
default implementation; it actually falls back to a Lucene<br>
implementation when this option is missing. This looks like IMHO<br>
adding some extra complexity into the class which you don't really<br>
need. Is there a good reason for that? Someone could forget some<br>
option in the configuration, it would be better to throw an exception<br>
to notify the user about the configuration inconsistency than to do<br>
something differently, or rely on a good default.<br>
Maybe I'm wrong, but then some comments could help me out. Is the user<br>
supposed to configure both message producers and receivers with the<br>
same kind of BackendQeueProcessorFactory? That's probably not needed,<br>
and not consistent with the way the JMS backend is configured.<br>
<br>
5)JGroupsAbstractMessageReceiver's design:<br>
This is very similar to the JMS abstract receiver, but in case of JMS<br>
I'd expect to have to "annotate" something in my ejb classes, so that<br>
it gets deployed by the container and associated to the queue, so in<br>
case of JMS it's mandatory for the user to write some class.<br>
Your solution is fine, but wouldn't it be possible to have a "no-code"<br>
solution? The user could just configure this deployment to say<br>
something like "this is the jgroups configuration, this is the<br>
hibernate configuration, you know where to find the entity classes...<br>
please listen to the channels and do your job".<br>
It would be very cool to have just to package the search jar with some<br>
configuration lines (and the entities of course to read some more<br>
Search configuration) and be ready to start listening for messages.<br>
Actually some future version could avoid the entities and receive the<br>
serialized configuration.. just a thought, but that would enable us to<br>
prepackage a whole server ready as a Search backend without even<br>
needing to deploy any user code.<br>
<br>
6) testing... I couldn't start them as JGroups was failing to bind to<br>
ports on my machine, I'm sure I am doing something wrong, will try<br>
again after reading some docs about it.<br>
But anyway I got a bit confused about the notion of "Master"s and<br>
"Receivers"; I'm used in the JMS to see the master as the one taking<br>
care of the index, so receiving the docs not sending them.<br>
<br>
Generally speaking, add some comments and debug log statements (using<br>
the {} instead of string + concatenation);<br>
I'll try this weekend to try it on remote staging servers, it looks promising!<br>
<br>
Sanne<br>
<br>
2009/6/10 £ukasz Moreñ <<a href="mailto:lukasz.moren@gmail.com">lukasz.moren@gmail.com</a>>:<br>
<div><div></div><div class="h5">> Hi,<br>
><br>
> I've finished task concerning JMS replacement with JGroups. The patch is<br>
> attached. The general idea of pushing indexes through JG is assured, however<br>
> there are issues to improve (i.e. flexible JG protocol stack configuration).<br>
> Any review or advices would be welcome to make sure that I am not going into<br>
> blind alley.<br>
><br>
> Thanks,<br>
> Lukasz<br>
><br>
> 2009/5/27 Emmanuel Bernard <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>><br>
>><br>
>> Lukasz,<br>
>> I have been discussing with Manik on #3 and we think that JBoss Cache /<br>
>> Infinispan are probably a better fit than plain JGroups for that as all the<br>
>> plumbing will be configured for you.<br>
>> When you reach this problem, let's revive this discussion.<br>
>><br>
>> On May 25, 2009, at 11:07, Hardy Ferentschik wrote:<br>
>><br>
>>> Hi,<br>
>>><br>
>>> I talked with £ukasz about this last wekk. Definitely, #1 and #3.<br>
>>> #2 I don't like either.<br>
>>><br>
>>> The befefit of #3 would also be that one could drop the requirement of<br>
>>> having a shared file system (NFS, NAS, ...) #3 should be quite easy to<br>
>>> implement. Maybe easy to get started with.<br>
>>><br>
>>> --Hardy<br>
>>><br>
>>> On Mon, 25 May 2009 10:55:52 +0200, Emmanuel Bernard<br>
>>> <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>> wrote:<br>
>>><br>
>>>> Hello<br>
>>>> I am not sure this is where we should go, or at least, it depends. here<br>
>>>> are three scenarii<br>
>>>><br>
>>>><br>
>>>> #1 JMS replacement<br>
>>>> If you want to use JGroups as a replacement for the JMS backend, then I<br>
>>>> think you should write a jgroups backend. Check<br>
>>>> org.hibernate.search.backend.impl.jms<br>
>>>> In this case all changes are sent via JGroups to a "master". The master<br>
>>>> could be voted by the cluster possibly dynamically but that's not necessary<br>
>>>> for the first version.<br>
>>>><br>
>>>> #2 apply indexing on all nodes<br>
>>>> JGroups could send the work queue to all nodes and each node could apply<br>
>>>> the change.<br>
>>>> for various reasons I am not fan of this solution as it creates overhead<br>
>>>> in CPU / memory usage and does nto scale very well from a theoretical PoV.<br>
>>>><br>
>>>> #3 Index copy<br>
>>>> this is what you are describing, copying the index using JGroups instead<br>
>>>> of my file system approach. This might have merits esp as we could diminish<br>
>>>> network traffic using multicast but it also require to rethink the master /<br>
>>>> slave modus operandi.<br>
>>>> Today the master copy on a regular basis a clean index to a shared<br>
>>>> directory<br>
>>>> On a regular basis, the slave go and copy the clean index from the<br>
>>>> shared directory.<br>
>>>> In your approach, the master would send changes to the slaves and slaves<br>
>>>> would have to apply them "right away" (on their passive version)<br>
>>>><br>
>>>> I think #1 is more interesting than #3, we probably should start with<br>
>>>> that. #3 might be interesting too, thoughts?<br>
>>>><br>
>>>> Emmanuel<br>
>>>><br>
>>>> PS: refactoring is a fact of life, so feel free to do so. Just don't<br>
>>>> break public contracts.<br>
>>>><br>
>>>> On May 21, 2009, at 22:14, £ukasz Moreñ wrote:<br>
>>>><br>
>>>>> Hi,<br>
>>>>><br>
>>>>> I have few questions that concern using JGroups to copy index files. I<br>
>>>>> think to create sender(for master) and receiver(slave) directory providers.<br>
>>>>> Sender class mainly based on existing FSMasterDirectoryProvider, first<br>
>>>>> create local index copy and send later to slave nodes<br>
>>>>> (or send without copying, but that may cause lower performance?).<br>
>>>>> To avoid code redundancy it would be good to refactor a little<br>
>>>>> FSMasterDirectoryProvider class, so then I can use copying functionality in<br>
>>>>> new DirectoryProvider and add sending one; or rather I should work around<br>
>>>>> it?<br>
>>>>><br>
>>>>> I do not understand completely how does the multithreading access to<br>
>>>>> index file work. Does FileChannel class assure that, when index is copied<br>
>>>>> and new Lucene works are pushed?<br>
>>><br>
>>><br>
>>><br>
>><br>
><br>
><br>
</div></div>> _______________________________________________<br>
> hibernate-dev mailing list<br>
> <a href="mailto:hibernate-dev@lists.jboss.org">hibernate-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/hibernate-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/hibernate-dev</a><br>
><br>
><br>
</blockquote></div><br>