[infinispan-dev] Ditching ASYNC modes for REPL/DIST/INV/CacheStores?

Wed Feb 26 05:30:41 EST 2014

On Wed, Feb 26, 2014 at 8:56 AM, Galder Zamarreño <galder at redhat.com> wrote:

>
> On 19 Feb 2014, at 12:03, Sanne Grinovero <sanne at infinispan.org> wrote:
>
> > On 19 February 2014 07:12, Galder Zamarreño <galder at redhat.com> wrote:
> >>
> >> On 03 Feb 2014, at 19:01, Dan Berindei <dan.berindei at gmail.com> wrote:
> >>
> >>>
> >>>
> >>>
> >>> On Mon, Feb 3, 2014 at 6:28 PM, Radim Vansa <rvansa at redhat.com> wrote:
> >>>>>>> For sync we would want to invoke directly to avoid context
> switching.
> >>>>>> I think you haven't properly understood what I was talking about:
> the
> >>>>>> putAsync should not switch context at all in the ideal design. It
> should
> >>>>>> traverse through the interceptors all the way down (logically, in
> >>>>>> current behaviour), invoke JGroups async API and jump out. Then, as
> soon
> >>>>>> as the response is received, the thread which delivered it should
> >>>>>> traverse the interceptor stack up (again, logically), and fire the
> future.
> >>>> A Future doesn't make much sense with an async transport.  The problem
> >>>> is with an async transport you never get back a response so you never
> >>>> know when the actual command is completed and thus a Future is
> >>>> worthless.  The caller wouldn't know if they could rely on the use of
> >>>> the Future or not.
> >>>
> >>> You're right, there's one important difference between putAsync and put
> >>> with async transport: in the first case you can find out when the
> >>> request is completed while you cannot with the latter. Not requiring
> the
> >>> ack can be an important optimization. I think that both versions are
> >>> very valid: first mostly for bulk operations = reduction of latency,
> >>> second for modifications that are acceptable to fail without handling
> that.
> >>> I had the first case in my mind when talking about async operations,
> and
> >>> there the futures are necessary.
> >>>
> >>> A couple more differences:
> >>> 1. You can't do commitAsync(), but you can configure the commit to be
> replicated asynchronously (1PC). Although we did talk about removing that
> option...
> >>> 2. If you do putAsync(k, v1); putAsync(k, v2), there is no ordering
> between the two and you might end up with k=v1 in the cache.
> >>
> >> If there's any relationship between both puts for the caller thread,
> the caller must make sure that the second put is only called after the
> first has completed.
> >
> > Actually in such a case I would strongly expect Infinispan to keep the
> > two operations in order. This is not to be pushed on user's
> > responsibility.
>
> If the two operations are executed by the same thread, then yes, I agree
> that it should be applied one after the other:
>
> Thread-1: Future f1 = putAsync(k, v1);
> Thread-1: Future f2 = putAsync(k, v2);
>
> I'd expect v1 to be applied and then v2. This operations would be added to
> some queue that you'd expect both insertions to happen one after the other,
> in Thread-1, so yeah, we can apply them in order.
>

This does definitely not happen at the moment in Infinispan. Each putAsync
gets its own asynchronous worker thread (there are 25 async threads by
default), and the threads are not synchronized in any way.

And I'm not sure it makes sense to order them anyway. I mean the order
between two sequential putAsync operations was preserved, it would be quite
natural to expect the ordering between a putAsync and a regular put to be
preserved as well.

Thread-1: Future f1 = putAsync(k, v1)
Thread-1: put(k, v2)
Thread-1: assert f1.isDone() && get(k).equals(v2)

This would get quite complicated... an async put always creates a new,
implicit, transaction, whereas a regular put can be part of an active
transaction. So preserving the ordering between the putAsync and the put
might mean delaying not the put, but the transaction commit.

I'm not saying this couldn't be done, but I'm not sure it would make the
semantics of putAsync any clearer than they are now.

>
> However, if the following happens:
>
> Thread-1: Future f1 = putAsync(k, v1);
> Thread-2: Future f2 = putAsync(k, v2);
>
> We can't be enforcing such ordering.
>
> Now, if there's a relationship to the eye of the beholder between v1 and
> v2, and you expect v2 to be the end result, this is how you'd have to do it
> (JDK8-esque):
>
> Thread-1: Future f1 = putAsync(k, v1);
> Thread-2: Future f2 = f1.map.putAsync(k, v2);
>
> or:
>
> Thread-1: Future f1 = putAsync(k, v1);
> Thread-2: Future f2 = f1.map.replaceAsync(k, v1, v2);
>

Do you mean here that the 2nd putAsync/the replaceAsync operation would
start executing only after f1 is done? Or would you expect them both to
start executing at once, but with Infinispan ensuring that the 2nd
operation is executed on the primary owner after the 1st?

If it's the former, it should be quite easy to implement a Future with a
getCache() method that returns a delegating cache, allowing you to submit a
put operation immediately, but blocking it until the future is done. If
it's the latter, I suspect it's going to be a lot more work.

> >
> >>
> >> If there's separate threads calling it and it relies on this, it should
> call replace the second time, i.e. replaceAsync(k, v1, v2) to get the
> guarantees it wants.
> >>
> >> What is really important is that the order in which they are executed
> in one node/replica is the same order in which they're executed in all
> other nodes. This was something that was not maintained when async
> marshalling was enabled.
> >
> > +1000
> >
> > But also I'd stress that any sync operation should have a Future
> > returned,
>
> ^ To me, purely sync operations are any operations that return anything
> other than a Future. IOW:
>
> void put(k, v);
>
> ^ That's an implicit sync operation where you have no choice.
>
> An async operation can behave both sync and async:
>
> Future<Void> put(k, v);
>
> Can be sync or async, depends on whether the user waits or does something
> once it completes. If it does not wait, or discards the Future, it's async.
> If it does somethign with the future, it's sync.
>

I don't agree with this. If the user can do something else while the
operation is executing, then the operation is async. I don't know if there
is a specific name in Java-land for starting an async call and discarding
the Future, but in .Net this pattern is called "fire-and-forget".

> > someone in this long thread suggested to have an option to
> > drop it for example to speedup bulk imports, but I really can't see a
> > scenario in which I wouldn't want to know about a failure.
>
> +1, I think everything should return a Future.
>

Even void put(k, v)??

>
> > Let's not
> > do the same mistake that made MongoDB so "popular" ;-)
> > Bulk imports can still be mad efficient without strictly needing to go
> > these lenghts.
> >
> > Sanne
> >
> >
> >>
> >>>
> >>>
> >>>>
> >>>> Also it depends what you are trying to do with async.  Currently async
> >>>> transport is only for sending messages to another node, we never think
> >>>> of when we are the owning node.  In this case the calling thread would
> >>>> have to go down the interceptor stack and acquire any locks if it is
> >>>> the owner, thus causing this "async" to block if you have any
> >>>> contention on the given key.  The use of another thread would allow
> >>>> the calling thread to be able to return immediately no matter what
> >>>> else is occurring.  Also I don't see what is so wrong about having a
> >>>> context switch to run something asynchronously, we shouldn't have a
> >>>> context switch to block the user thread imo, which is very possible
> >>>> with locking.
> >>>
> >>> This is an important notice! Locking would complicate the design a lot,
> >>> because the thread in "async" mode should do only tryLocks - if this
> >>> fails, further processing should be dispatched to another thread. Not
> >>> sure if this could be implemented at all, because the thread may be
> >>> blocked inside JGroups as well (async API is about receiving the
> >>> response asynchronously, not about sending the message asynchronously).
> >>>
> >>> I don't say that the context switch is that bad. My concern is that you
> >>> have a very limited amount of requests that can be processed in
> >>> parallel. I consider a "request" something pretty lightweight in
> concept
> >>> - but one thread per request makes this rather heavyweight stuff.
> >>>
> >>> We did talk in Farnborough/Palma about removing the current
> LockManager with a queue-based structure like the one used for ordering
> total-order transactions. And about removing the implicit stack in the
> current interceptor stack with an explicit stack, to allow resuming a
> command mid-execution. But the feeling I got was that neither is going to
> make it into 7.0.
> >>>
> >>>
> >>>>
> >>>>> +1 much cleaner, I love it. Actually wasn't aware the current code
> >>>>> didn't do this :-(
> >>>> This is what the current async transport does, but it does nothing
> with Futures.
> >>>
> >>> Nevermind the futures, this is not the important part. It's not about
> >>> async transport neither, it's about async executors.
> >>> (okay, the thread was about dropping async transport, I have hijacked
> it)
> >>>
> >>> Radim
> >>>
> >>> --
> >>> Radim Vansa <rvansa at redhat.com>
> >>> JBoss DataGrid QA
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >>
> >> --
> >> Galder Zamarreño
> >> galder at redhat.com
> >> twitter.com/galderz
> >>
> >> Project Lead, Escalante
> >> http://escalante.io
> >>
> >> Engineer, Infinispan
> >> http://infinispan.org
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarreño
> galder at redhat.com
> twitter.com/galderz
>
> Project Lead, Escalante
> http://escalante.io
>
> Engineer, Infinispan
> http://infinispan.org
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140226/3228b679/attachment-0001.html 