<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 14, 2013 at 9:32 AM, Radim Vansa <span dir="ltr"><<a href="mailto:rvansa@redhat.com" target="_blank">rvansa@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Blocking OOB threads is the thing we want to avoid, remember?<br></blockquote><div><br></div><div>Well, you have to block somewhere...<br>
<br>I like Adrian's solution, because it's a lot better than CallerRunsPolicy: it's blocking the OOB thread until any other command finishes executing, not until one particular command finishes executing.<br>
<br>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I am used do synchronous replication, there we could hold a semaphore on requests to each node. Going down before each RPC and up after that would be pretty simple and having this mechanism in RPC layer it would block only request, not response messages. </blockquote>
<div><br></div><div>I don't think you can throttle on the sender, because you don't know how many threads the recipient should allocate per sender in the Infinispan thread pool. <br><br>E.g. in non-tx concurrent mode, for each user-initiated request, the primary owner sends out numOwners requests to the other nodes. If the cache is also replicated, you have the coordinator sending clusterSize requests to the other nodes for each user-initiated request. (So for each request that a node receives from the coordinator, it will receive exactly 0 requests from the other nodes.)<br>
<br></div><div>If you start considering that each cache should have its own semaphore, because they need different numbers of threads, then finding an appropriate initial value is pretty much impossible.<br><br></div><div>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Are there any shoot & forget messages sent from Infinispan which could not be applied in this way?<br>
Regarding the async replication, I am not sure whether the request is just processed in non-caller thrad or if there is anything more.<br>
<span class=""><font color="#888888"><br></font></span></blockquote><div><br></div><div>Async messages require ordering most of the time, so they're handled on the Incoming thread pool directly and they're not passed to the Infinispan thread pool (at least for now).<br>
</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class=""><font color="#888888">
Radim<br>
</font></span><div class=""><div class="h5"><br>
----- Original Message -----<br>
| From: "Adrian Nistor" <<a href="mailto:anistor@redhat.com">anistor@redhat.com</a>><br>
| To: "infinispan -Dev List" <<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a>><br>
| Sent: Wednesday, March 13, 2013 3:37:23 PM<br>
| Subject: Re: [infinispan-dev] DefaultExecutorFactory and rejection policy<br>
|<br>
| What about creating a RejectedExecutionHandler that blocks the<br>
| calling<br>
| thread if the thread pool has reached the max (and all threads are<br>
| busy)<br>
| and the associated queue is also full? See<br>
| org.hibernate.search.batchindexing.impl.Executors.BlockPolicy<br>
|<br>
| Some time ago I was thinking we need this for infinispan's thread<br>
| pools<br>
| too because of some very insidious test suite failures caused by<br>
| dropped<br>
| tasks and created ISPN-2438 but never got to experiment with this.<br>
| The<br>
| solution used for the moment was to increase the pool a bit for unit<br>
| tests...<br>
|<br>
| Cheers<br>
|<br>
| On 03/13/2013 04:18 PM, Pedro Ruivo wrote:<br>
| > By default, the thread pools are bounded and we have nothing to<br>
| > prevent<br>
| > the it to be full.<br>
| ><br>
| > currently, the regular message are not processed by the thread<br>
| > pool.<br>
| > this can be solved when a smarter dispatcher/ordering mechanism is<br>
| > implemented.<br>
| ><br>
| > On 03/13/2013 02:07 PM, Bela Ban wrote:<br>
| >> You mean an unbounded queue ? This would be bad if the insertion<br>
| >> rate of<br>
| >> messages is higher than the processing rate.<br>
| >><br>
| >> The problem with the Infinispan pool is that we cannot simply<br>
| >> discard<br>
| >> messages; they won't get retransmitted, as they're been delivered<br>
| >> by<br>
| >> JGroups.<br>
| >><br>
| >> If we end up with our own ordering of messages in the ISPN pool,<br>
| >> then a<br>
| >> rejection policy of "run" would also cause harm, as it would<br>
| >> destroy the<br>
| >> ordering (except for OOB messages)...<br>
| >><br>
| >> On 3/13/13 2:52 PM, Erik Salter wrote:<br>
| >>> Any reason the messages can't be queued? In my mind, the<br>
| >>> benefits would<br>
| >>> outweigh the unbounded nature of it.<br>
| >>><br>
| >>> Erik<br>
| >>><br>
| >>> -----Original Message-----<br>
| >>> From: <a href="mailto:infinispan-dev-bounces@lists.jboss.org">infinispan-dev-bounces@lists.jboss.org</a><br>
| >>> [mailto:<a href="mailto:infinispan-dev-bounces@lists.jboss.org">infinispan-dev-bounces@lists.jboss.org</a>] On Behalf Of<br>
| >>> Pedro Ruivo<br>
| >>> Sent: Wednesday, March 13, 2013 9:29 AM<br>
| >>> To: ispn-dev<br>
| >>> Subject: [infinispan-dev] DefaultExecutorFactory and rejection<br>
| >>> policy<br>
| >>><br>
| >>> Hi<br>
| >>><br>
| >>> I'm working on ISPN-2808<br>
| >>> (<a href="https://issues.jboss.org/browse/ISPN-2808" target="_blank">https://issues.jboss.org/browse/ISPN-2808</a>) and I<br>
| >>> noticed that the DefaultExecutorFactory is creating the executor<br>
| >>> service<br>
| >>> with an Abortpolicy.<br>
| >>><br>
| >>> Is there any particular reason for that?<br>
| >>><br>
| >>> In the new thread pool needed by ISPN-2808, I cannot have the<br>
| >>> messages (i.e.<br>
| >>> the runnables) discarded, because it can cause some inconsistent<br>
| >>> state and<br>
| >>> even block all the cluster.<br>
| >>><br>
| >>> I have set in my branch a CallerRunPolicy. If you see any issue<br>
| >>> with this<br>
| >>> let me know.<br>
| >>><br>
| >>> Cheers,<br>
| >>> Pedro<br>
| >><br>
| > _______________________________________________<br>
| > infinispan-dev mailing list<br>
| > <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
| > <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
|<br>
| _______________________________________________<br>
| infinispan-dev mailing list<br>
| <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
| <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
|<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</div></div></blockquote></div><br></div></div>