Hey Radim, Sorry for late reply. Indeed, I think you should start by getting RC done first. When I first experimented with this, I had bigger issues than the ones explained here. For example, how to pass down transaction context when multiple async operations are being executed within a transaction. In particular, say you have two async operations, chained, executed with T1 and T2 respectively, both part of the the same transaction. Suspending tx in T1 and resuming it when T2 starts is easy, the problem I had was how to get T1 to resume the transaction when T2 has completed async transactional operation. At the time I created a forum post in the transactions forum about it [1]. I was not able to able to try out the solution in [1], but it doesn't feel very idiomatic to me (e.g. needing to call CompletableFuture.get()). Did you consider this issue? Or are you avoiding it somehow? Cheers, [1] https://developer.jboss.org/thread/265595 -- Galder Zamarreño Infinispan, Red Hat > On 27 Sep 2016, at 16:51, Radim Vansa <rvansa(a)redhat.com&gt; wrote: > > Hi, > > seems I'll have to implement the functional stuff on tx caches [1][2] if > I want to get rid of DeltaAware et al. > > The general idea is quite simple - ReadOnly* commands should behave very > similar to non-tx mode, WriteOnly* commands will be just added as > modifications to the PrepareCommand and ReadWrite* commands will be both > added to modifications list, and sent to remote nodes where the result > won't be stored yet. > The results of operations should not be stored into transactional > context - the command will execute remotely (if the owners are remote) > unless the value was read by Get* beforehand. > > With repeatable-reads isolation, the situation gets more complicated. If > we use ReadOnly* that performs identity lookup (effectively the same as > Get*) and the entry was modified in during the transaction, we can > return two different results - so a read committed semantics. With write > skew check enabled, we could at least fail the transaction at the end > (the check would be performed reads as well if the transaction contains > functional reads), but we cannot rely on WSC always on with RR. > > Retrieving the whole entry and applying the functional command is not a > viable solution, IMO - that would completely defy the purpose of using > functional command. > > A possible solution would be to send the global transaction ID with > those read commands and keep a remote transactional context with read > entries for the duration of transaction on remote nodes, too. However, > if we do a Read* command to primary owner, it's possible that further > Get* command will hit backup. So, we could go to all owners with Read* > already during the transaction (slowing down functional reads > considerably), or read only from primary owner (which slows down Get*s > even if we don't use functional APIs - this makes it a no-go). I am not > 100% sure how a transaction transfer during ST will get into that. > > We could also do it the ostrich way - "Yes we've promissed RR but Func > will be only RC". I'll probably do that in the first draft anyway. > > Comments & opinions appreciated. > > Radim > > [1] https://issues.jboss.org/browse/ISPN-5806 > [2] https://issues.jboss.org/browse/ISPN-6573 > > -- > Radim Vansa <rvansa(a)redhat.com&gt; > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

In theory something like this should work, resuming the transaction after each asynchronous operation: https://paste.fedoraproject.org/461451/77496416/raw/ I'm not sure about the usability, or performance... The TransactionManager API doesn't allow asynchronous commit(), so one might as well bite the bullet and spawn a new thread to run their transaction in.

A more pie-in-the-sky idea for the the functional API would be to support "batching" multiple evals over the same cache, not involving the transaction manager. That should allow us to catch WriteSkewExceptions and retry the batch ourselves, assuming the lambdas don't have side-effects. And if the transaction is Infinispan-only, we can also make the commit asynchronous.

There's always the option of forcing the functional operations to be synchronous when they're joining a running TransactionManager transaction... Cheers Dan On Tue, Oct 11, 2016 at 8:16 PM, Radim Vansa <rvansa(a)redhat.com&gt; wrote: > Hi Galder, > > I've looked to the Narayana thread but I also have some trouble > understanding the R ret = CompletableFuture.supplyAsync(...) - that > method returns CF<R>, not directly R... so the answer that guys on the > forum give you makes sense to me. Maybe you could revive that thread and > clarify what you had in mind. > > You're describing general issue of async calls, not just functional API. > Functional API is about moving some lambda to be executed in situ. Your > question is no different as if you were calling putAsync and trying to > chain it with another operations. TBH I haven't dealt with this problem > at all, the tests I've written use the operations in sync way. > > If you try to do something transactionally-sensitive as a part of the > lambda itself, you're doing it wrong - the lambda can be executed on > different node which has no link to the original transaction. And the > lambda should be purely functional (no side effects), otherwise you're > poking the devil. > > Regarding RC vs. RR., in [1] I've decided to attach version on which the > functional command operated with the return value if versions (and WSC) > is used. Then, if the version changes during the transaction (say, > you've did a functional read, and it changes before second read), > WriteSkewException is thrown as we're not operating on the same value. > However, this does not work after you modify the entry with a functional > command; then your reads still work on the non-modified entry... I have > to tweak this a bit yet, basically sending all the modifications to a > given entry along with the read :-/ > Note: RR without WSC is just broken, I should add a warning log somewhere... > > Radim > > [1] https://github.com/infinispan/infinispan/pull/4608 > > On 10/10/2016 05:49 PM, Galder Zamarreño wrote: >> Hey Radim, >> >> Sorry for late reply. Indeed, I think you should start by getting RC done first. >> >> When I first experimented with this, I had bigger issues than the ones explained here. For example, how to pass down transaction context when multiple async operations are being executed within a transaction. >> >> In particular, say you have two async operations, chained, executed with T1 and T2 respectively, both part of the the same transaction. Suspending tx in T1 and resuming it when T2 starts is easy, the problem I had was how to get T1 to resume the transaction when T2 has completed async transactional operation. At the time I created a forum post in the transactions forum about it [1]. I was not able to able to try out the solution in [1], but it doesn't feel very idiomatic to me (e.g. needing to call CompletableFuture.get()). >> >> Did you consider this issue? Or are you avoiding it somehow? >> >> Cheers, >> >> [1] https://developer.jboss.org/thread/265595 >> -- >> Galder Zamarreño >> Infinispan, Red Hat >> >>> On 27 Sep 2016, at 16:51, Radim Vansa <rvansa(a)redhat.com&gt; wrote: >>> >>> Hi, >>> >>> seems I'll have to implement the functional stuff on tx caches [1][2] if >>> I want to get rid of DeltaAware et al. >>> >>> The general idea is quite simple - ReadOnly* commands should behave very >>> similar to non-tx mode, WriteOnly* commands will be just added as >>> modifications to the PrepareCommand and ReadWrite* commands will be both >>> added to modifications list, and sent to remote nodes where the result >>> won't be stored yet. >>> The results of operations should not be stored into transactional >>> context - the command will execute remotely (if the owners are remote) >>> unless the value was read by Get* beforehand. >>> >>> With repeatable-reads isolation, the situation gets more complicated. If >>> we use ReadOnly* that performs identity lookup (effectively the same as >>> Get*) and the entry was modified in during the transaction, we can >>> return two different results - so a read committed semantics. With write >>> skew check enabled, we could at least fail the transaction at the end >>> (the check would be performed reads as well if the transaction contains >>> functional reads), but we cannot rely on WSC always on with RR. >>> >>> Retrieving the whole entry and applying the functional command is not a >>> viable solution, IMO - that would completely defy the purpose of using >>> functional command. >>> >>> A possible solution would be to send the global transaction ID with >>> those read commands and keep a remote transactional context with read >>> entries for the duration of transaction on remote nodes, too. However, >>> if we do a Read* command to primary owner, it's possible that further >>> Get* command will hit backup. So, we could go to all owners with Read* >>> already during the transaction (slowing down functional reads >>> considerably), or read only from primary owner (which slows down Get*s >>> even if we don't use functional APIs - this makes it a no-go). I am not >>> 100% sure how a transaction transfer during ST will get into that. >>> >>> We could also do it the ostrich way - "Yes we've promissed RR but Func >>> will be only RC". I'll probably do that in the first draft anyway. >>> >>> Comments & opinions appreciated. >>> >>> Radim >>> >>> [1] https://issues.jboss.org/browse/ISPN-5806 >>> [2] https://issues.jboss.org/browse/ISPN-6573 >>> >>> -- >>> Radim Vansa <rvansa(a)redhat.com&gt; >>> JBoss Performance Team >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Radim Vansa <rvansa(a)redhat.com&gt; > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 27/10/16 09:36, Radim Vansa wrote: > So it's probably a question to Narayana guys if you can suspend a > transaction in one thread and resume it in two threads. yes you can. Where it gets hairy is ensuring all the threads are done before you commit. By default we warn rather than block, though there is a (non-standard) plugin api to register your own handler for it.

Jonathan.