It's a bit complex but the Lucene IndexWriter works better if you can
inject what has to be indexed in a multi threaded way because
analyzing the data before writing it to the index can be heavy.
The IW does rely on what is called a Directory (implemented by
Infinispan in this case).
On 13 août 09, at 17:25, Jason T. Greene wrote:
Any reason you cant do a transaction per thread? That would be the
most efficient.
Łukasz Moreń wrote:
> Newly created threads were not associated with any transaction, so I
> suppose it was a problem. Sharing transaction between threads seems
> to
> be a good solution.
> Thanks for help!
> 2009/8/13, Jason T. Greene <jason.greene(a)redhat.com>:
>> Correct. Also there could be read races as well, so if you are
>> going to
>> share a tx between threads, i would use some shared lock to
>> gaurantee
>> that only one thread can use it at a time. BTW this means you have
>> to
>> properly suspend/resume the TX via the TM API as well.
>>
>> Emmanuel Bernard wrote:
>>> Modifying a transaction means applying muations (like SQL INSERT /
>>> UPDATE / DELETE) to the transactional resource?
>>>
>>> On 13 août 09, at 15:07, Jason T. Greene wrote:
>>>
>>>> When using transactions, the context is bound to the
>>>> transaction, and
>>>> you can move a transaction between threads. However, you should
>>>> only
>>>> be modifying a transaction with one thread at a time.
>>>>
>>>> Emmanuel Bernard wrote:
>>>>> Could it be that you are not using the same transaction between
>>>>> different threads (ie you physically start different ones or
>>>>> different "Infinispan contexts")?
>>>>> Infini guys, do you support transactional operation spanning
>>>>> several
>>>>> concurrent threads?
>>>>> On 13 août 09, at 14:04, Łukasz Moreń wrote:
>>>>>> I've tried with JBoss AS transaction manager and
>>>>>> JBossStandaloneTM.
>>>>>> The result is this same in all cases - error during merge.
>>>>>>
>>>>>> 2009/8/12, Emmanuel Bernard <emmanuel(a)hibernate.org>:
>>>>>>> Ok I understand better now.
>>>>>>> Do your tests in JBoss AS with it's decent transaction
manager
>>>>>>> (infinispan should have a config for it)
>>>>>>> For unit testing, force the indexing process in hibernate to
>>>>>>> use a
>>>>>>> single thread (I ghnk it's possible ask Sanne of you
don't
>>>>>>> know how).
>>>>>>>
>>>>>>> Exposing some configuration to infinispan makes sense. can
you
>>>>>>> start a
>>>>>>> thread explainig what is configurable and which one you think
>>>>>>> we
>>>>>>> should expose to hsearch users. Ideally I would like to offer
>>>>>>> one or
>>>>>>> two defaut config scenarios and allow to fallback to a custom
>>>>>>> config.
>>>>>>>
>>>>>>> Emmanuel
>>>>>>>
>>>>>>> On 12 août 2009, at 11:58, Łukasz Moreń
>>>>>>> <lukasz.moren(a)gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Sorry, but my wifi does not work well today. I will try
to
>>>>>>>> explain
>>>>>>>> it more clear.
>>>>>>>>
>>>>>>>> I'm using DummyTransactionManager available for
Infinispan.
>>>>>>>> It associates transaction with the calling thread.
>>>>>>>>
>>>>>>>> Steps to update index:
>>>>>>>>
>>>>>>>> 1. index writer acquires lock - begin of transaction
>>>>>>>>
>>>>>>>> 2. if it is necessary, index writer delegates new threads
to
>>>>>>>> do
>>>>>>>> merge work.
>>>>>>>> Those merge threads do not see changes made so far from
>>>>>>>> begin of
>>>>>>>> transaction,
>>>>>>>> and are looking for segments which are not yet in index.
>>>>>>>> Changes will be visible when AD.3 is completed.
>>>>>>>> For tests i tried to commit transaction when merge starts
>>>>>>>> and then
>>>>>>>> everything worked well. But then i need to start it
again.
>>>>>>>>
>>>>>>>> 3. index writer releases lock - transaction is commited,
all
>>>>>>>> changes
>>>>>>>> made in this transaction are visible for other threads.
>>>>>>>>
>>>>>>>> Maybe using some other transaction manager could help?
>>>>>>>>
>>>>>>>> What about Infinispan cache configuration? Some
configuration
>>>>>>>> mechanism should be exposed to the user,
>>>>>>>> or we can hardcoded one in InfinispanDirectoryProvider is
>>>>>>>> enough?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2009/8/12 Emmanuel Bernard
<emmanuel(a)hibernate.org>
>>>>>>>> why?
>>>>>>>> Emmanuel Bernard
>>>>>>>> Pending
>>>>>>>> you there?
>>>>>>>> Emmanuel Bernard
>>>>>>>> Pending
>>>>>>>> Ok please describe in details what is going on. From what
>>>>>>>> you are
>>>>>>>> describing the tx cannot see all segments which looks
like an
>>>>>>>> infinispan bug to me.
>>>>>>>> Pending
>>>>>>>>
>>>>>>>> As a back up you can try wo transaction and see if that
works
>>>>>>>> Emmanuel Bernard
>>>>>>>> Pending
>>>>>>>> technically the lucene index should cope with that
>>>>>>>> Emmanuel Bernard
>>>>>>>> 11:16
>>>>>>>> but I like this approach less
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Let's try and chat by email IF I'm not online, I
need to run
>>>>>>>> on some
>>>>>>>> errands today.
>>>>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev(a)lists.jboss.org
>>>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Jason T. Greene
>>>> JBoss, a division of Red Hat
>>
>> --
>> Jason T. Greene
>> JBoss, a division of Red Hat
>>
--
Jason T. Greene
JBoss, a division of Red Hat