[infinispan-dev] Ditching ASYNC modes for REPL/DIST/INV/CacheStores?

Galder Zamarreño galder at redhat.com
Wed Feb 26 01:56:08 EST 2014 

On 19 Feb 2014, at 12:03, Sanne Grinovero <sanne at infinispan.org> wrote:

> On 19 February 2014 07:12, Galder Zamarreño <galder at redhat.com> wrote:
>> 
>> On 03 Feb 2014, at 19:01, Dan Berindei <dan.berindei at gmail.com> wrote:
>> 
>>> 
>>> 
>>> 
>>> On Mon, Feb 3, 2014 at 6:28 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>>> For sync we would want to invoke directly to avoid context switching.
>>>>>> I think you haven't properly understood what I was talking about: the
>>>>>> putAsync should not switch context at all in the ideal design. It should
>>>>>> traverse through the interceptors all the way down (logically, in
>>>>>> current behaviour), invoke JGroups async API and jump out. Then, as soon
>>>>>> as the response is received, the thread which delivered it should
>>>>>> traverse the interceptor stack up (again, logically), and fire the future.
>>>> A Future doesn't make much sense with an async transport.  The problem
>>>> is with an async transport you never get back a response so you never
>>>> know when the actual command is completed and thus a Future is
>>>> worthless.  The caller wouldn't know if they could rely on the use of
>>>> the Future or not.
>>> 
>>> You're right, there's one important difference between putAsync and put
>>> with async transport: in the first case you can find out when the
>>> request is completed while you cannot with the latter. Not requiring the
>>> ack can be an important optimization. I think that both versions are
>>> very valid: first mostly for bulk operations = reduction of latency,
>>> second for modifications that are acceptable to fail without handling that.
>>> I had the first case in my mind when talking about async operations, and
>>> there the futures are necessary.
>>> 
>>> A couple more differences:
>>> 1. You can't do commitAsync(), but you can configure the commit to be replicated asynchronously (1PC). Although we did talk about removing that option...
>>> 2. If you do putAsync(k, v1); putAsync(k, v2), there is no ordering between the two and you might end up with k=v1 in the cache.
>> 
>> If there’s any relationship between both puts for the caller thread, the caller must make sure that the second put is only called after the first has completed.
> 
> Actually in such a case I would strongly expect Infinispan to keep the
> two operations in order. This is not to be pushed on user's
> responsibility.

If the two operations are executed by the same thread, then yes, I agree that it should be applied one after the other:

Thread-1: Future f1 = putAsync(k, v1);
Thread-1: Future f2 = putAsync(k, v2);

I’d expect v1 to be applied and then v2. This operations would be added to some queue that you’d expect both insertions to happen one after the other, in Thread-1, so yeah, we can apply them in order.

However, if the following happens:

Thread-1: Future f1 = putAsync(k, v1);
Thread-2: Future f2 = putAsync(k, v2);

We can’t be enforcing such ordering.

Now, if there’s a relationship to the eye of the beholder between v1 and v2, and you expect v2 to be the end result, this is how you’d have to do it (JDK8-esque):

Thread-1: Future f1 = putAsync(k, v1);
Thread-2: Future f2 = f1.map.putAsync(k, v2);

or:

Thread-1: Future f1 = putAsync(k, v1);
Thread-2: Future f2 = f1.map.replaceAsync(k, v1, v2);

> 
>> 
>> If there’s separate threads calling it and it relies on this, it should call replace the second time, i.e. replaceAsync(k, v1, v2) to get the guarantees it wants.
>> 
>> What is really important is that the order in which they are executed in one node/replica is the same order in which they’re executed in all other nodes. This was something that was not maintained when async marshalling was enabled.
> 
> +1000
> 
> But also I'd stress that any sync operation should have a Future
> returned,

^ To me, purely sync operations are any operations that return anything other than a Future. IOW:

void put(k, v);

^ That’s an implicit sync operation where you have no choice.

An async operation can behave both sync and async:

Future<Void> put(k, v);

Can be sync or async, depends on whether the user waits or does something once it completes. If it does not wait, or discards the Future, it’s async. If it does somethign with the future, it’s sync.

> someone in this long thread suggested to have an option to
> drop it for example to speedup bulk imports, but I really can't see a
> scenario in which I wouldn't want to know about a failure.

+1, I think everything should return a Future.

> Let's not
> do the same mistake that made MongoDB so "popular" ;-)
> Bulk imports can still be mad efficient without strictly needing to go
> these lenghts.
> 
> Sanne
> 
> 
>> 
>>> 
>>> 
>>>> 
>>>> Also it depends what you are trying to do with async.  Currently async
>>>> transport is only for sending messages to another node, we never think
>>>> of when we are the owning node.  In this case the calling thread would
>>>> have to go down the interceptor stack and acquire any locks if it is
>>>> the owner, thus causing this "async" to block if you have any
>>>> contention on the given key.  The use of another thread would allow
>>>> the calling thread to be able to return immediately no matter what
>>>> else is occurring.  Also I don't see what is so wrong about having a
>>>> context switch to run something asynchronously, we shouldn't have a
>>>> context switch to block the user thread imo, which is very possible
>>>> with locking.
>>> 
>>> This is an important notice! Locking would complicate the design a lot,
>>> because the thread in "async" mode should do only tryLocks - if this
>>> fails, further processing should be dispatched to another thread. Not
>>> sure if this could be implemented at all, because the thread may be
>>> blocked inside JGroups as well (async API is about receiving the
>>> response asynchronously, not about sending the message asynchronously).
>>> 
>>> I don't say that the context switch is that bad. My concern is that you
>>> have a very limited amount of requests that can be processed in
>>> parallel. I consider a "request" something pretty lightweight in concept
>>> - but one thread per request makes this rather heavyweight stuff.
>>> 
>>> We did talk in Farnborough/Palma about removing the current LockManager with a queue-based structure like the one used for ordering total-order transactions. And about removing the implicit stack in the current interceptor stack with an explicit stack, to allow resuming a command mid-execution. But the feeling I got was that neither is going to make it into 7.0.
>>> 
>>> 
>>>> 
>>>>> +1 much cleaner, I love it. Actually wasn't aware the current code
>>>>> didn't do this :-(
>>>> This is what the current async transport does, but it does nothing with Futures.
>>> 
>>> Nevermind the futures, this is not the important part. It's not about
>>> async transport neither, it's about async executors.
>>> (okay, the thread was about dropping async transport, I have hijacked it)
>>> 
>>> Radim
>>> 
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss DataGrid QA
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> --
>> Galder Zamarreño
>> galder at redhat.com
>> twitter.com/galderz
>> 
>> Project Lead, Escalante
>> http://escalante.io
>> 
>> Engineer, Infinispan
>> http://infinispan.org
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org

More information about the infinispan-dev mailing list