[infinispan-dev] [hibernate-dev] Infinispan tx, config and multithreading

Thu Aug 13 18:24:30 EDT 2009

Yes, right, MergeSchedulers.

2009/8/14 Sanne Grinovero <sanne.grinovero at gmail.com>

> what are these "other" threads? Are you speaking about the MergeSchedulers?
>
> 2009/8/13 Łukasz Moreń <lukasz.moren at gmail.com>:
> > IndexWriter processes index update and delegates some job to other
> > threads and waits when they finish. These "other" threads works on
> > data modified
> > in IndexWriter transaction. So I think if I use transaction per
> > thread, "others" would not see data modified by IndexWriter until
> > commit.
> >
> > 2009/8/13, Emmanuel Bernard <emmanuel at hibernate.org>:
> >> Ah I thought it was using multiple threads because of your mass
> >> indexing. I did not know some threads were span specifically for the
> >> Infinispan directory.
> >>
> >> On 13 août 09, at 17:34, Sanne Grinovero wrote:
> >>
> >>> Hi Łukasz,
> >>> what is your usage of these threads? did you consider using one
> >>> transaction per thread?
> >>>
> >>> Sanne
> >>>
> >>> 2009/8/13 Łukasz Moreń <lukasz.moren at gmail.com>:
> >>>> Newly created threads were not associated with any transaction, so I
> >>>> suppose it was a problem. Sharing transaction between threads seems
> >>>> to
> >>>> be a good solution.
> >>>> Thanks for help!
> >>>>
> >>>> 2009/8/13, Jason T. Greene <jason.greene at redhat.com>:
> >>>>> Correct. Also there could be read races as well, so if you are
> >>>>> going to
> >>>>> share a tx between threads, i would use some shared lock to
> >>>>> gaurantee
> >>>>> that only one thread can use it at a time. BTW this means you have
> >>>>> to
> >>>>> properly suspend/resume the TX via the TM API as well.
> >>>>>
> >>>>> Emmanuel Bernard wrote:
> >>>>>> Modifying a transaction means applying muations (like SQL INSERT /
> >>>>>> UPDATE / DELETE) to the transactional resource?
> >>>>>>
> >>>>>> On 13 août 09, at 15:07, Jason T. Greene wrote:
> >>>>>>
> >>>>>>> When using transactions, the context is bound to the
> >>>>>>> transaction, and
> >>>>>>> you can move a transaction between threads. However, you should
> >>>>>>> only
> >>>>>>> be modifying a transaction with one thread at a time.
> >>>>>>>
> >>>>>>> Emmanuel Bernard wrote:
> >>>>>>>> Could it be that you are not using the same transaction between
> >>>>>>>> different threads (ie you physically start different ones or
> >>>>>>>> different  "Infinispan contexts")?
> >>>>>>>> Infini guys, do you support transactional operation spanning
> >>>>>>>> several
> >>>>>>>> concurrent threads?
> >>>>>>>> On 13 août 09, at 14:04, Łukasz Moreń wrote:
> >>>>>>>>> I've tried with JBoss AS transaction manager and
> >>>>>>>>> JBossStandaloneTM.
> >>>>>>>>> The result is this same in all cases - error during merge.
> >>>>>>>>>
> >>>>>>>>> 2009/8/12, Emmanuel Bernard <emmanuel at hibernate.org>:
> >>>>>>>>>> Ok I understand better now.
> >>>>>>>>>> Do your tests in JBoss AS with it's decent transaction manager
> >>>>>>>>>> (infinispan should have a config for it)
> >>>>>>>>>> For unit testing, force the indexing process in hibernate to
> >>>>>>>>>> use a
> >>>>>>>>>> single thread (I ghnk it's possible ask Sanne of you don't
> >>>>>>>>>> know how).
> >>>>>>>>>>
> >>>>>>>>>> Exposing some configuration to infinispan makes sense. can you
> >>>>>>>>>> start a
> >>>>>>>>>> thread explainig what is configurable and which one you think
> >>>>>>>>>> we
> >>>>>>>>>> should expose to hsearch users. Ideally I would like to offer
> >>>>>>>>>> one or
> >>>>>>>>>> two defaut config scenarios and allow to fallback to a custom
> >>>>>>>>>> config.
> >>>>>>>>>>
> >>>>>>>>>> Emmanuel
> >>>>>>>>>>
> >>>>>>>>>> On 12 août 2009, at 11:58, Łukasz Moreń
> >>>>>>>>>> <lukasz.moren at gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Sorry, but my wifi does not work well today. I will try to
> >>>>>>>>>>> explain
> >>>>>>>>>>> it more clear.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm using DummyTransactionManager available for Infinispan.
> >>>>>>>>>>> It associates transaction with the calling thread.
> >>>>>>>>>>>
> >>>>>>>>>>> Steps to update index:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. index writer acquires lock - begin of transaction
> >>>>>>>>>>>
> >>>>>>>>>>> 2. if it is necessary, index writer delegates new threads to
> >>>>>>>>>>> do
> >>>>>>>>>>> merge work.
> >>>>>>>>>>> Those merge threads do not see changes made so far from
> >>>>>>>>>>> begin of
> >>>>>>>>>>> transaction,
> >>>>>>>>>>> and are looking for segments which are not yet in index.
> >>>>>>>>>>> Changes will be visible when AD.3 is completed.
> >>>>>>>>>>> For tests i tried to commit transaction when merge starts
> >>>>>>>>>>> and then
> >>>>>>>>>>> everything worked well. But then i need to start it again.
> >>>>>>>>>>>
> >>>>>>>>>>> 3. index writer releases lock - transaction is commited, all
> >>>>>>>>>>> changes
> >>>>>>>>>>> made in this transaction are visible for other threads.
> >>>>>>>>>>>
> >>>>>>>>>>> Maybe using some other transaction manager could help?
> >>>>>>>>>>>
> >>>>>>>>>>> What about Infinispan cache configuration? Some configuration
> >>>>>>>>>>> mechanism should be exposed to the user,
> >>>>>>>>>>> or we can hardcoded one in InfinispanDirectoryProvider is
> >>>>>>>>>>> enough?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2009/8/12 Emmanuel Bernard <emmanuel at hibernate.org>
> >>>>>>>>>>> why?
> >>>>>>>>>>> Emmanuel Bernard
> >>>>>>>>>>> Pending
> >>>>>>>>>>> you there?
> >>>>>>>>>>> Emmanuel Bernard
> >>>>>>>>>>> Pending
> >>>>>>>>>>> Ok please describe in details what is going on. From what
> >>>>>>>>>>> you are
> >>>>>>>>>>> describing the tx cannot see all segments which looks like an
> >>>>>>>>>>> infinispan bug to me.
> >>>>>>>>>>> Pending
> >>>>>>>>>>>
> >>>>>>>>>>> As a back up you can try wo transaction and see if that works
> >>>>>>>>>>> Emmanuel Bernard
> >>>>>>>>>>> Pending
> >>>>>>>>>>> technically the lucene index should cope with that
> >>>>>>>>>>> Emmanuel Bernard
> >>>>>>>>>>> 11:16
> >>>>>>>>>>> but I like this approach less
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Let's try and chat by email IF I'm not online, I need to run
> >>>>>>>>>>> on some
> >>>>>>>>>>> errands today.
> >>>>>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> infinispan-dev mailing list
> >>>>>>>> infinispan-dev at lists.jboss.org
> >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Jason T. Greene
> >>>>>>> JBoss, a division of Red Hat
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Jason T. Greene
> >>>>> JBoss, a division of Red Hat
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>> _______________________________________________
> >>> hibernate-dev mailing list
> >>> hibernate-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20090814/7f6563dc/attachment-0002.html 