[hibernate-dev] [infinispan-dev] Infinispan tx, config and multithreading

[Emmanuel Bernard]

It's a bit complex but the Lucene IndexWriter works better if you can  
inject what has to be indexed in a multi threaded way because  
analyzing the data before writing it to the index can be heavy.
The IW does rely on what is called a Directory (implemented by  
Infinispan in this case).

On 13 août 09, at 17:25, Jason T. Greene wrote:

> Any reason you cant do a transaction per thread? That would be the  
> most efficient.
>
> Łukasz Moreń wrote:
>> Newly created threads were not associated with any transaction, so I
>> suppose it was a problem. Sharing transaction between threads seems  
>> to
>> be a good solution.
>> Thanks for help!
>> 2009/8/13, Jason T. Greene <jason.greene at redhat.com>:
>>> Correct. Also there could be read races as well, so if you are  
>>> going to
>>> share a tx between threads, i would use some shared lock to  
>>> gaurantee
>>> that only one thread can use it at a time. BTW this means you have  
>>> to
>>> properly suspend/resume the TX via the TM API as well.
>>>
>>> Emmanuel Bernard wrote:
>>>> Modifying a transaction means applying muations (like SQL INSERT /
>>>> UPDATE / DELETE) to the transactional resource?
>>>>
>>>> On 13 août 09, at 15:07, Jason T. Greene wrote:
>>>>
>>>>> When using transactions, the context is bound to the  
>>>>> transaction, and
>>>>> you can move a transaction between threads. However, you should  
>>>>> only
>>>>> be modifying a transaction with one thread at a time.
>>>>>
>>>>> Emmanuel Bernard wrote:
>>>>>> Could it be that you are not using the same transaction between
>>>>>> different threads (ie you physically start different ones or
>>>>>> different  "Infinispan contexts")?
>>>>>> Infini guys, do you support transactional operation spanning  
>>>>>> several
>>>>>> concurrent threads?
>>>>>> On 13 août 09, at 14:04, Łukasz Moreń wrote:
>>>>>>> I've tried with JBoss AS transaction manager and  
>>>>>>> JBossStandaloneTM.
>>>>>>> The result is this same in all cases - error during merge.
>>>>>>>
>>>>>>> 2009/8/12, Emmanuel Bernard <emmanuel at hibernate.org>:
>>>>>>>> Ok I understand better now.
>>>>>>>> Do your tests in JBoss AS with it's decent transaction manager
>>>>>>>> (infinispan should have a config for it)
>>>>>>>> For unit testing, force the indexing process in hibernate to  
>>>>>>>> use a
>>>>>>>> single thread (I ghnk it's possible ask Sanne of you don't  
>>>>>>>> know how).
>>>>>>>>
>>>>>>>> Exposing some configuration to infinispan makes sense. can you
>>>>>>>> start a
>>>>>>>> thread explainig what is configurable and which one you think  
>>>>>>>> we
>>>>>>>> should expose to hsearch users. Ideally I would like to offer  
>>>>>>>> one or
>>>>>>>> two defaut config scenarios and allow to fallback to a custom  
>>>>>>>> config.
>>>>>>>>
>>>>>>>> Emmanuel
>>>>>>>>
>>>>>>>> On 12 août 2009, at 11:58, Łukasz Moreń  
>>>>>>>> <lukasz.moren at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Sorry, but my wifi does not work well today. I will try to  
>>>>>>>>> explain
>>>>>>>>> it more clear.
>>>>>>>>>
>>>>>>>>> I'm using DummyTransactionManager available for Infinispan.
>>>>>>>>> It associates transaction with the calling thread.
>>>>>>>>>
>>>>>>>>> Steps to update index:
>>>>>>>>>
>>>>>>>>> 1. index writer acquires lock - begin of transaction
>>>>>>>>>
>>>>>>>>> 2. if it is necessary, index writer delegates new threads to  
>>>>>>>>> do
>>>>>>>>> merge work.
>>>>>>>>> Those merge threads do not see changes made so far from  
>>>>>>>>> begin of
>>>>>>>>> transaction,
>>>>>>>>> and are looking for segments which are not yet in index.
>>>>>>>>> Changes will be visible when AD.3 is completed.
>>>>>>>>> For tests i tried to commit transaction when merge starts  
>>>>>>>>> and then
>>>>>>>>> everything worked well. But then i need to start it again.
>>>>>>>>>
>>>>>>>>> 3. index writer releases lock - transaction is commited, all  
>>>>>>>>> changes
>>>>>>>>> made in this transaction are visible for other threads.
>>>>>>>>>
>>>>>>>>> Maybe using some other transaction manager could help?
>>>>>>>>>
>>>>>>>>> What about Infinispan cache configuration? Some configuration
>>>>>>>>> mechanism should be exposed to the user,
>>>>>>>>> or we can hardcoded one in InfinispanDirectoryProvider is  
>>>>>>>>> enough?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2009/8/12 Emmanuel Bernard <emmanuel at hibernate.org>
>>>>>>>>> why?
>>>>>>>>> Emmanuel Bernard
>>>>>>>>> Pending
>>>>>>>>> you there?
>>>>>>>>> Emmanuel Bernard
>>>>>>>>> Pending
>>>>>>>>> Ok please describe in details what is going on. From what  
>>>>>>>>> you are
>>>>>>>>> describing the tx cannot see all segments which looks like an
>>>>>>>>> infinispan bug to me.
>>>>>>>>> Pending
>>>>>>>>>
>>>>>>>>> As a back up you can try wo transaction and see if that works
>>>>>>>>> Emmanuel Bernard
>>>>>>>>> Pending
>>>>>>>>> technically the lucene index should cope with that
>>>>>>>>> Emmanuel Bernard
>>>>>>>>> 11:16
>>>>>>>>> but I like this approach less
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Let's try and chat by email IF I'm not online, I need to run  
>>>>>>>>> on some
>>>>>>>>> errands today.
>>>>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>> --
>>>>> Jason T. Greene
>>>>> JBoss, a division of Red Hat
>>>
>>> --
>>> Jason T. Greene
>>> JBoss, a division of Red Hat
>>>
>
>
> -- 
> Jason T. Greene
> JBoss, a division of Red Hat