Hi Manik, all this, aggregating and chaining Futures looks like the way to go, but what about streaming ? It would be elegant to think of a completely 'streamed' interface, using Pipes for the asynchronous aspect (a Piped(In/Out)putStream has to be running in another thread -> the Future's thread ?). Combining Futures and Pipes sound like an even better solution, lower memory usage and higher throughput ! Being able to stream data in and out of the cache is particularly interesting for a Qi4j entitystore since we also use marshaling (JSon for the moment). And JClouds has a streamed interface, we could connect all the streams together (Infinispan extended Map interface -> Marshaling -> JClouds) and fill multiple pipes at the same time.... sounds like a dream, a funny one ? Cheers phil On Wed, Jan 13, 2010 at 3:22 PM, Manik Surtani <manik(a)jboss.org&gt; wrote: So I've been spending some time thinking about how we deal with async tasks in Infinispan, both from an API perspective as well as an implementation detail, and wanted to throw a few ideas out there. First, lets understand the 4 most expensive things that happen in Infinispan, either simply expensive or things that could block under high contention (in descending order): RPC calls, marshalling, CacheStore and locking. We deal with asynchronism in a somewhat haphazard way at the moment, each of these functions receiving a somewhat different treatment: 1) RPC: Using JGroups' ResponseMode of waiting for none. 2) Marshalling: using an async repl executor to take this offline 3) Use an AsyncStore wrapper which places tasks in an executor 4) Nothing and to add to it, 5) READs are never asynchronous. E.g., no such thing as an async GET - even if it entails RPC or a CacheStore lookup (which may be a remote call like S3!) The impact of this approach is that end users never really get to benefit from the general asynchronism in place, and still needs to configure stuff in several different places. And internally, it makes dealing with internal APIs hard. Externally, this is quite well encapsulated in the Cache interface, by offering async methods such as putAsync(), etc. so there would be little to change here. They return Futures, parameterized to the actual method return type, e.g., putAsync returns Future<V> in a parameterized Cache<K, V>. (More precisely, they return a NotifyingFuture, a sub-interface of Future that allows attaching listeners, but that's a detail.) So I think we should start with that. The user receives a Future. This Future is an aggregate Future, which should aggregate and block based on several sub-Futures, one for each of the tasks (1 ~ 4) outlined above. Now what is the impact of this? Designing such a Future is easy enough, but how would this change internal components? 1) RPC. Async RPC is, IMO, broken at the moment. It is unsafe in that it offers no guarantees that the calls are received by the recipient, and you have no way of knowing. So RPC should always be synchronous, but wrapped in a Future so that it is taken offline and the Future can be checked for success. 2) Marshalling happens offline at the moment and a Future is returned as it stands, but this could probably be combined with RPC to a single Future since this step is logically before RPC, and RPC relies on marshalling to complete. 3) The CacheStore interface should only return parameterized Futures. Implementations that do not offer any async support may simply return a wrapped value. Calls to this interface that require immediate results would simply poll the Future. But in all other cases - ranging from the AsyncStore wrapper to more sophisticated async backends such as JClouds, we get to harness the async nature offered by the impl. 4) Locking could be async, but understanding context is important here. We currently use 2 different types of Locks: JDK reentrant locks for non-transactional calls, or OwnableReentrantLock for transactional calls where the lock owner is not the thread but instead the GlobalTransaction. For this to work, any invocation context should have reference to the original invoking thread, OwnableReentrantLocks should always be used, and the owner set accordingly, to ensure an Executor thread doesn't gain ownership of a lock instead. The end result of all this, is that async API calls will be truly non-blocking, and will be able t harness as much parallelism as possible in the system. Another effect is that we'd have a cleaner internal API which is specifically designed with async operations in mind, and converting any of the async calls to sync calls is trivial (Future.get()) anywhere along the invocation chain. Also, I think once we have this in place, any async transport modes (REPL_ASYNC, DIST_ASYNC) should be deprecated in favour of an async API, since you get the ability to check that a call completes even though it happens offline. And a disclaimer: I have no plans to implement anything like this soon, I was just throwing this out for comment and distillation. :) Cheers -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Dnia ¶roda 13 styczeñ 2010 o 15:22:01 napisa³e¶: > 1) RPC: Using JGroups' ResponseMode of waiting for none. > Speaking of which... I ran into a problem in JBoss Cache some time ago (it's one of those not yet reported ;) ) with aynchronous replication and non-blocking state transfer. AFAICS Infinispan uses the same replication mechanism as JBC, so I think it's relevant here. The thing is that the "asynchronous" replication uses the usual method of submitting a task to an Executor. This is no problem without NBST, because the tasks are not interested in responses and finish immediately, but with NBSP they need responses to facilitate replay. And since they need to wait for responses, they clog the Executor's threads and under sufficiently heavy load, it all degrades to effectively synchronous replication. And it happens quite fast, really. The good news is that I fixed it. The bad news is that it's not something that can really be put in public view :) But the approach is to use asynchronous API in JGroups - one of the castMessage() overloads that accepts a RspCollector. That way I made the async replication really asynchronous, and so far it works without problems :) -KS

So dude, let's recap: 1. You found an issue but didn't report it 2. You fixed the issue but cannot talk about it What's next ? You'd have to kill us if you talked about the issue and the fix ? :-)

> And my previous mail might have been a little rushed as I sent it just > before going home in the evening. Do you want me to report it properly? Yes, we would appreciate it if you could report it for JBoss Cache. I can then create a similar, linked issue for Infinispan and if you do have a patch, we could look at that too - even if we don't use it in its entireity.

So I've been spending some time thinking about how we deal with async tasks in Infinispan, both from an API perspective as well as an implementation detail, and wanted to throw a few ideas out there. First, lets understand the 4 most expensive things that happen in Infinispan, either simply expensive or things that could block under high contention (in descending order): RPC calls, marshalling, CacheStore and locking

We deal with asynchronism in a somewhat haphazard way at the moment, each of these functions receiving a somewhat different treatment: 1) RPC: Using JGroups' ResponseMode of waiting for none. 2) Marshalling: using an async repl executor to take this offline

3) Use an AsyncStore wrapper which places tasks in an executor

4) Nothing and to add to it, 5) READs are never asynchronous. E.g., no such thing as an async GET - even if it entails RPC or a CacheStore lookup (which may be a remote call like S3!) The impact of this approach is that end users never really get to benefit from the general asynchronism in place

and still needs to configure stuff in several different places. And internally, it makes dealing with internal APIs hard. Externally, this is quite well encapsulated in the Cache interface, by offering async methods such as putAsync(), etc. so there would be little to change here. They return Futures, parameterized to the actual method return type, e.g., putAsync returns Future<V> in a parameterized Cache<K, V>. (More precisely, they return a NotifyingFuture, a sub-interface of Future that allows attaching listeners, but that's a detail.) So I think we should start with that. The user receives a Future. This Future is an aggregate Future, which should aggregate and block based on several sub-Futures, one for each of the tasks (1 ~ 4) outlined above. Now what is the impact of this? Designing such a Future is easy enough, but how would this change internal components?

1) RPC. Async RPC is, IMO, broken at the moment. It is unsafe in that it offers no guarantees that the calls are received by the recipient

and you have no way of knowing.

So RPC should always be synchronous, but wrapped in a Future so that it is taken offline and the Future can be checked for success

2) Marshalling happens offline at the moment and a Future is returned as it stands, but this could probably be combined with RPC to a single Future since this step is logically before RPC, and RPC relies on marshalling to complete

On 13 Jan 2010, at 17:13, Bela Ban wrote: > > > Manik Surtani wrote: >> So I've been spending some time thinking about how we deal with async >> tasks in Infinispan, both from an API perspective as well as an >> implementation detail, and wanted to throw a few ideas out there. >> >> First, lets understand the 4 most expensive things that happen in >> Infinispan, either simply expensive or things that could block under >> high contention (in descending order): RPC calls, marshalling, >> CacheStore and locking > > > In my experience, RPC calls (at least async ones) should be much less > costly than CacheStore and *un*-marshalling (marshalling *is* usually > fast). Re CacheStore: writing / reading to a disk is slower than even a > network round trip. Well, it depends on the case. Network saturation, load and which handles concurrency better. But either way, expensive things that can and should be parallelized. > We deal with asynchronism in a somewhat haphazard way at the moment, >> each of these functions receiving a somewhat different treatment: >> >> 1) RPC: Using JGroups' ResponseMode of waiting for none. >> 2) Marshalling: using an async repl executor to take this offline > > The problem here is that you're pushing the problem of marshalling > further down the line. *Eventually* data has to be marshalled, and > somebody *has* to block ! IIRC, you used a bounded queue to place > marshalling tasks onto, so for load peaks this was fine, but for > constant high load, someone will always block on the (full) queue. That's where it happens right now. Just before RPC. Now perhaps there is some sense in understanding that more than one subsystem may need a marshalled representation of an entry (e.g., the RpcManager to push across the wire, as well as a CacheStore to persist to disk or network again), so this could happen prior to either of these calls. And also useful to note - as you mention later - that UNmarshalling is often far slower than marshalling, so any sync network calls should not wait for entries to be unmarshalled on the remote end. We provide for this to some degree with the lazyDeserialization config element which makes use of MarshalledValues. It could just be better integrated with the rest of what we are doing re: async.

On 01/15/2010 03:09 AM, Galder Zamarreno wrote:> > > On 01/13/2010 06:56 PM, Manik Surtani wrote: >> >> On 13 Jan 2010, at 17:13, Bela Ban wrote: >>> We deal with asynchronism in a somewhat haphazard way at the moment, >>>> each of these functions receiving a somewhat different treatment: >>>> >>>> 1) RPC: Using JGroups' ResponseMode of waiting for none. >>>> 2) Marshalling: using an async repl executor to take this offline >>> >>> The problem here is that you're pushing the problem of marshalling >>> further down the line. *Eventually* data has to be marshalled, and >>> somebody *has* to block ! IIRC, you used a bounded queue to place >>> marshalling tasks onto, so for load peaks this was fine, but for >>> constant high load, someone will always block on the (full) queue. >> >> That's where it happens right now. Just before RPC. Now perhaps there is some sense in understanding that more than one subsystem may need a marshalled representation of an entry (e.g., the RpcManager to push across the wire, as well as a CacheStore to persist to disk or network again), so this could happen prior to either of these calls. And also useful to note - as you mention later - that UNmarshalling is often far slower than marshalling, so any sync network calls should not wait for entries to be unmarshalled on the remote end. We provide for this to some degree with the lazyDeserialization config element which makes use of MarshalledValues. It could just be better integrated with the rest of what we are doing re: async. > > Note that there was also some talk in the past about using > MarshalledValues all the time in a email thread called "storing in > memory data in binary format" in the infinispan-dev list. This enabled > memory based eviction policies. > Configurable please. I don't want Infinispan storing the data I put in it in binary form unless I tell it to.

On 01/13/2010 08:22 AM, Manik Surtani wrote: > The end result of all this, is that async API calls will be truly non-blocking, and will be able t harness as much parallelism as possible in the system. Another effect is that we'd have a cleaner internal API which is specifically designed with async operations in mind, and converting any of the async calls to sync calls is trivial (Future.get()) anywhere along the invocation chain. Also, I think once we have this in place, any async transport modes (REPL_ASYNC, DIST_ASYNC) should be deprecated in favour of an async API, since you get the ability to check that a call completes even though it happens offline. > Please don't remove xx_ASYNC. If I use your async API, I will in most cases throw away the Future. Forcing JGroups to respond to the RPC and correlate the responses is a waste of resources.

-- Brian Stansberry Lead, AS Clustering JBoss by Red Hat _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev