Yes, right, MergeSchedulers.<br><br><div class="gmail_quote">2009/8/14 Sanne Grinovero <span dir="ltr"><<a href="mailto:sanne.grinovero@gmail.com">sanne.grinovero@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
what are these "other" threads? Are you speaking about the MergeSchedulers?<br>
<div><div></div><div class="h5"><br>
2009/8/13 Łukasz Moreń <<a href="mailto:lukasz.moren@gmail.com">lukasz.moren@gmail.com</a>>:<br>
> IndexWriter processes index update and delegates some job to other<br>
> threads and waits when they finish. These "other" threads works on<br>
> data modified<br>
> in IndexWriter transaction. So I think if I use transaction per<br>
> thread, "others" would not see data modified by IndexWriter until<br>
> commit.<br>
><br>
> 2009/8/13, Emmanuel Bernard <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>>:<br>
>> Ah I thought it was using multiple threads because of your mass<br>
>> indexing. I did not know some threads were span specifically for the<br>
>> Infinispan directory.<br>
>><br>
>> On 13 août 09, at 17:34, Sanne Grinovero wrote:<br>
>><br>
>>> Hi Łukasz,<br>
>>> what is your usage of these threads? did you consider using one<br>
>>> transaction per thread?<br>
>>><br>
>>> Sanne<br>
>>><br>
>>> 2009/8/13 Łukasz Moreń <<a href="mailto:lukasz.moren@gmail.com">lukasz.moren@gmail.com</a>>:<br>
>>>> Newly created threads were not associated with any transaction, so I<br>
>>>> suppose it was a problem. Sharing transaction between threads seems<br>
>>>> to<br>
>>>> be a good solution.<br>
>>>> Thanks for help!<br>
>>>><br>
>>>> 2009/8/13, Jason T. Greene <<a href="mailto:jason.greene@redhat.com">jason.greene@redhat.com</a>>:<br>
>>>>> Correct. Also there could be read races as well, so if you are<br>
>>>>> going to<br>
>>>>> share a tx between threads, i would use some shared lock to<br>
>>>>> gaurantee<br>
>>>>> that only one thread can use it at a time. BTW this means you have<br>
>>>>> to<br>
>>>>> properly suspend/resume the TX via the TM API as well.<br>
>>>>><br>
>>>>> Emmanuel Bernard wrote:<br>
>>>>>> Modifying a transaction means applying muations (like SQL INSERT /<br>
>>>>>> UPDATE / DELETE) to the transactional resource?<br>
>>>>>><br>
>>>>>> On 13 août 09, at 15:07, Jason T. Greene wrote:<br>
>>>>>><br>
>>>>>>> When using transactions, the context is bound to the<br>
>>>>>>> transaction, and<br>
>>>>>>> you can move a transaction between threads. However, you should<br>
>>>>>>> only<br>
>>>>>>> be modifying a transaction with one thread at a time.<br>
>>>>>>><br>
>>>>>>> Emmanuel Bernard wrote:<br>
>>>>>>>> Could it be that you are not using the same transaction between<br>
>>>>>>>> different threads (ie you physically start different ones or<br>
>>>>>>>> different "Infinispan contexts")?<br>
>>>>>>>> Infini guys, do you support transactional operation spanning<br>
>>>>>>>> several<br>
>>>>>>>> concurrent threads?<br>
>>>>>>>> On 13 août 09, at 14:04, Łukasz Moreń wrote:<br>
>>>>>>>>> I've tried with JBoss AS transaction manager and<br>
>>>>>>>>> JBossStandaloneTM.<br>
>>>>>>>>> The result is this same in all cases - error during merge.<br>
>>>>>>>>><br>
>>>>>>>>> 2009/8/12, Emmanuel Bernard <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>>:<br>
>>>>>>>>>> Ok I understand better now.<br>
>>>>>>>>>> Do your tests in JBoss AS with it's decent transaction manager<br>
>>>>>>>>>> (infinispan should have a config for it)<br>
>>>>>>>>>> For unit testing, force the indexing process in hibernate to<br>
>>>>>>>>>> use a<br>
>>>>>>>>>> single thread (I ghnk it's possible ask Sanne of you don't<br>
>>>>>>>>>> know how).<br>
>>>>>>>>>><br>
>>>>>>>>>> Exposing some configuration to infinispan makes sense. can you<br>
>>>>>>>>>> start a<br>
>>>>>>>>>> thread explainig what is configurable and which one you think<br>
>>>>>>>>>> we<br>
>>>>>>>>>> should expose to hsearch users. Ideally I would like to offer<br>
>>>>>>>>>> one or<br>
>>>>>>>>>> two defaut config scenarios and allow to fallback to a custom<br>
>>>>>>>>>> config.<br>
>>>>>>>>>><br>
>>>>>>>>>> Emmanuel<br>
>>>>>>>>>><br>
>>>>>>>>>> On 12 août 2009, at 11:58, Łukasz Moreń<br>
>>>>>>>>>> <<a href="mailto:lukasz.moren@gmail.com">lukasz.moren@gmail.com</a>><br>
>>>>>>>>>> wrote:<br>
>>>>>>>>>><br>
>>>>>>>>>>> Sorry, but my wifi does not work well today. I will try to<br>
>>>>>>>>>>> explain<br>
>>>>>>>>>>> it more clear.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> I'm using DummyTransactionManager available for Infinispan.<br>
>>>>>>>>>>> It associates transaction with the calling thread.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Steps to update index:<br>
>>>>>>>>>>><br>
>>>>>>>>>>> 1. index writer acquires lock - begin of transaction<br>
>>>>>>>>>>><br>
>>>>>>>>>>> 2. if it is necessary, index writer delegates new threads to<br>
>>>>>>>>>>> do<br>
>>>>>>>>>>> merge work.<br>
>>>>>>>>>>> Those merge threads do not see changes made so far from<br>
>>>>>>>>>>> begin of<br>
>>>>>>>>>>> transaction,<br>
>>>>>>>>>>> and are looking for segments which are not yet in index.<br>
>>>>>>>>>>> Changes will be visible when AD.3 is completed.<br>
>>>>>>>>>>> For tests i tried to commit transaction when merge starts<br>
>>>>>>>>>>> and then<br>
>>>>>>>>>>> everything worked well. But then i need to start it again.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> 3. index writer releases lock - transaction is commited, all<br>
>>>>>>>>>>> changes<br>
>>>>>>>>>>> made in this transaction are visible for other threads.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Maybe using some other transaction manager could help?<br>
>>>>>>>>>>><br>
>>>>>>>>>>> What about Infinispan cache configuration? Some configuration<br>
>>>>>>>>>>> mechanism should be exposed to the user,<br>
>>>>>>>>>>> or we can hardcoded one in InfinispanDirectoryProvider is<br>
>>>>>>>>>>> enough?<br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>> 2009/8/12 Emmanuel Bernard <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>><br>
>>>>>>>>>>> why?<br>
>>>>>>>>>>> Emmanuel Bernard<br>
>>>>>>>>>>> Pending<br>
>>>>>>>>>>> you there?<br>
>>>>>>>>>>> Emmanuel Bernard<br>
>>>>>>>>>>> Pending<br>
>>>>>>>>>>> Ok please describe in details what is going on. From what<br>
>>>>>>>>>>> you are<br>
>>>>>>>>>>> describing the tx cannot see all segments which looks like an<br>
>>>>>>>>>>> infinispan bug to me.<br>
>>>>>>>>>>> Pending<br>
>>>>>>>>>>><br>
>>>>>>>>>>> As a back up you can try wo transaction and see if that works<br>
>>>>>>>>>>> Emmanuel Bernard<br>
>>>>>>>>>>> Pending<br>
>>>>>>>>>>> technically the lucene index should cope with that<br>
>>>>>>>>>>> Emmanuel Bernard<br>
>>>>>>>>>>> 11:16<br>
>>>>>>>>>>> but I like this approach less<br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>> Let's try and chat by email IF I'm not online, I need to run<br>
>>>>>>>>>>> on some<br>
>>>>>>>>>>> errands today.<br>
>>>>>>>>>>><br>
>>>>>>>> _______________________________________________<br>
>>>>>>>> infinispan-dev mailing list<br>
>>>>>>>> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
>>>>>>>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> --<br>
>>>>>>> Jason T. Greene<br>
>>>>>>> JBoss, a division of Red Hat<br>
>>>>>><br>
>>>>><br>
>>>>><br>
>>>>> --<br>
>>>>> Jason T. Greene<br>
>>>>> JBoss, a division of Red Hat<br>
>>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> infinispan-dev mailing list<br>
>>>> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
>>>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>>><br>
>>> _______________________________________________<br>
>>> hibernate-dev mailing list<br>
>>> <a href="mailto:hibernate-dev@lists.jboss.org">hibernate-dev@lists.jboss.org</a><br>
>>> <a href="https://lists.jboss.org/mailman/listinfo/hibernate-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/hibernate-dev</a><br>
>><br>
>><br>
><br>
</div></div></blockquote></div><br>