Recently we have been moving a lot of our methods that return Future [1] to CompletableFuture [2]. Unfortunately the latter, CompletableFuture, doesn't allow for cancellation of the future, since there is no thread tied to it. So I am proposing that our DistributedExecutorService [3] no longer allows for interruption of remote threads on a cancellation. This way we can have our distributed executor service return CompletableFuture instances which do not support interruption. Also to note that DistributedExecutorService extends ExecutorService which returns a normal Future which is documented as allowing cancellation. In this case I would just document on DistributedExecutorService that we don't support interruption anyways. Does anyone require the use of interruptable tasks with Distributed Executor? Thanks, - Will [1] https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html [2] https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Completabl... [3] https://docs.jboss.org/infinispan/8.1/apidocs/org/infinispan/distexec/Dis... _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I agree with Sanne, we need cancellable tasks. Tristan On 30/03/2016 18:19, Sanne Grinovero wrote: > The term "Interruption" might have been too specific, but being able > to cancel a task seems essential to me. > > On 30 March 2016 at 17:04, William Burns <mudokonman(a)gmail.com&gt; wrote: >> Recently we have been moving a lot of our methods that return Future [1] to >> CompletableFuture [2]. Unfortunately the latter, CompletableFuture, doesn't >> allow for cancellation of the future, since there is no thread tied to it. >> So I am proposing that our DistributedExecutorService [3] no longer allows >> for interruption of remote threads on a cancellation. This way we can have >> our distributed executor service return CompletableFuture instances which do >> not support interruption. >> >> Also to note that DistributedExecutorService extends ExecutorService which >> returns a normal Future which is documented as allowing cancellation. In >> this case I would just document on DistributedExecutorService that we don't >> support interruption anyways. >> >> Does anyone require the use of interruptable tasks with Distributed >> Executor? >> >> Thanks, >> >> - Will >> >> [1] >> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html >> [2] >> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Completabl... >> [3] >> https://docs.jboss.org/infinispan/8.1/apidocs/org/infinispan/distexec/Dis... >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 30 March 2016 at 17:40, William Burns <mudokonman(a)gmail.com&gt; wrote: > You can still attempt to cancel a task. This email is specifically about > interruption though. Let me explain the nuances in a bit more detail. > > With the suggestion we would still fully support if the task hasn't yet > started that it would be cancellable. > > However when you cancel a task, there is an mayInterruptIfRunning boolean. > If mayInterruptIfRunning is true and the task is already running it will try > to interrupt the thread processing it. This is what this email trail is > about. We all know that java interruption can be flaky to begin with and > then adding a remote aspect to it, it becomes even more unreliable. > > So do you guys think we need to support "interrupting" a task in progress? Yes, absolutely! Resources are precious :) See also the reasoning of our old friend: - https://corner.squareup.com/2016/01/query-sniper.html Take the example of a non-correctly tuned Hibernate Search "rebuild all indexes" task. On a non-trivial data set it might need to run for weeks... you need a way to kill it.

Thanks, Sanne > > - Will > > On Wed, Mar 30, 2016 at 12:29 PM Tristan Tarrant <ttarrant(a)redhat.com&gt; > wrote: >> >> I agree with Sanne, we need cancellable tasks. >> >> Tristan >> >> On 30/03/2016 18:19, Sanne Grinovero wrote: >> > The term "Interruption" might have been too specific, but being able >> > to cancel a task seems essential to me. >> > >> > On 30 March 2016 at 17:04, William Burns <mudokonman(a)gmail.com&gt; wrote: >> >> Recently we have been moving a lot of our methods that return Future >> >> [1] to >> >> CompletableFuture [2]. Unfortunately the latter, CompletableFuture, >> >> doesn't >> >> allow for cancellation of the future, since there is no thread tied to >> >> it. >> >> So I am proposing that our DistributedExecutorService [3] no longer >> >> allows >> >> for interruption of remote threads on a cancellation. This way we can >> >> have >> >> our distributed executor service return CompletableFuture instances >> >> which do >> >> not support interruption. >> >> >> >> Also to note that DistributedExecutorService extends ExecutorService >> >> which >> >> returns a normal Future which is documented as allowing cancellation. >> >> In >> >> this case I would just document on DistributedExecutorService that we >> >> don't >> >> support interruption anyways. >> >> >> >> Does anyone require the use of interruptable tasks with Distributed >> >> Executor? >> >> >> >> Thanks, >> >> >> >> - Will >> >> >> >> [1] >> >> >> >> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html >> >> [2] >> >> >> >> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Completabl... >> >> [3] >> >> >> >> https://docs.jboss.org/infinispan/8.1/apidocs/org/infinispan/distexec/Dis... >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> >> -- >> Tristan Tarrant >> Infinispan Lead >> JBoss, a division of Red Hat >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Thanks for the feedback. So in that case to try to appease both worlds: interrupting tasks and using CompletableFuture, I will just document the DistributedExecutorService to say that the mayInterruptIfRunning argument is only used on the returned future instance from methods such as submit and it is ignored on any chained CompletableFutures. - Will On Wed, Mar 30, 2016 at 1:37 PM Gustavo Fernandes <gustavo(a)infinispan.org <mailto:gustavo@infinispan.org>> wrote: On Wed, Mar 30, 2016 at 6:26 PM, Sanne Grinovero <sanne(a)infinispan.org <mailto:sanne@infinispan.org>> wrote: On 30 March 2016 at 17:40, William Burns <mudokonman(a)gmail.com <mailto:mudokonman@gmail.com>> wrote: > You can still attempt to cancel a task. This email is specifically about > interruption though. Let me explain the nuances in a bit more detail. > > With the suggestion we would still fully support if the task hasn't yet > started that it would be cancellable. > > However when you cancel a task, there is an mayInterruptIfRunning boolean. > If mayInterruptIfRunning is true and the task is already running it will try > to interrupt the thread processing it. This is what this email trail is > about. We all know that java interruption can be flaky to begin with and > then adding a remote aspect to it, it becomes even more unreliable. > > So do you guys think we need to support "interrupting" a task in progress? Yes, absolutely! Resources are precious :) See also the reasoning of our old friend: - https://corner.squareup.com/2016/01/query-sniper.html Take the example of a non-correctly tuned Hibernate Search "rebuild all indexes" task. On a non-trivial data set it might need to run for weeks... you need a way to kill it. That's the case of our Mass Indexer: one of the motivations of making it return a Future was the possibility of cancelling tasks, capability offered by the Distributed Executor. Gustavo Thanks, Sanne > > - Will > > On Wed, Mar 30, 2016 at 12:29 PM Tristan Tarrant <ttarrant(a)redhat.com <mailto:ttarrant@redhat.com>> > wrote: >> >> I agree with Sanne, we need cancellable tasks. >> >> Tristan >> >> On 30/03/2016 18:19, Sanne Grinovero wrote: >> > The term "Interruption" might have been too specific, but being able >> > to cancel a task seems essential to me. >> > >> > On 30 March 2016 at 17:04, William Burns <mudokonman(a)gmail.com <mailto:mudokonman@gmail.com>> wrote: >> >> Recently we have been moving a lot of our methods that return Future >> >> [1] to >> >> CompletableFuture [2]. Unfortunately the latter, CompletableFuture, >> >> doesn't >> >> allow for cancellation of the future, since there is no thread tied to >> >> it. >> >> So I am proposing that our DistributedExecutorService [3] no longer >> >> allows >> >> for interruption of remote threads on a cancellation. This way we can >> >> have >> >> our distributed executor service return CompletableFuture instances >> >> which do >> >> not support interruption. >> >> >> >> Also to note that DistributedExecutorService extends ExecutorService >> >> which >> >> returns a normal Future which is documented as allowing cancellation. >> >> In >> >> this case I would just document on DistributedExecutorService that we >> >> don't >> >> support interruption anyways. >> >> >> >> Does anyone require the use of interruptable tasks with Distributed >> >> Executor? >> >> >> >> Thanks, >> >> >> >> - Will >> >> >> >> [1] >> >> >> >> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html >> >> [2] >> >> >> >> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Completabl... >> >> [3] >> >> >> >> https://docs.jboss.org/infinispan/8.1/apidocs/org/infinispan/distexec/Dis... >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> >> -- >> Tristan Tarrant >> Infinispan Lead >> JBoss, a division of Red Hat >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On Thu, Mar 31, 2016 at 10:24 AM, Radim Vansa <rvansa(a)redhat.com&gt; wrote: > Can any of the chained futures be already running when the first is not > completed? Because when it's not running, there's nothing to interrupt. It's not a matter of timing, but the CF instance returned by any CF method will not pass the cancellation request to the initial future. So this code won't cancel anything: CompletableFuture<Void> cf = CompletableFuture.allOf(des.submitEverywhere(() -> null).toArray(new CompletableFuture[0])); cf.cancel(true); > > Btw., if the interface returns CompletableFuture but you inherit from > that to override the cancel method, the javadoc will still say that the > mayInterruptIfRunning does not apply. In that case, you should either > return InterruptableCompletableFuture with javadoc override, or maybe > just AnotherFuture implements Future, CompletionStage (names at your > discretion). Yeah, we should also make it clear that remote tasks are interrupted -- we know that's how it used to work, but it wasn't documented. > > R. > > On 03/30/2016 08:13 PM, William Burns wrote: >> Thanks for the feedback. >> >> So in that case to try to appease both worlds: interrupting tasks and >> using CompletableFuture, I will just document the >> DistributedExecutorService to say that the mayInterruptIfRunning >> argument is only used on the returned future instance from methods >> such as submit and it is ignored on any chained CompletableFutures. >> >> - Will >> >> On Wed, Mar 30, 2016 at 1:37 PM Gustavo Fernandes >> <gustavo(a)infinispan.org <mailto:gustavo@infinispan.org>> wrote: >> >> On Wed, Mar 30, 2016 at 6:26 PM, Sanne Grinovero >> <sanne(a)infinispan.org <mailto:sanne@infinispan.org>> wrote: >> >> On 30 March 2016 at 17:40, William Burns <mudokonman(a)gmail.com >> <mailto:mudokonman@gmail.com>> wrote: >> > You can still attempt to cancel a task. This email is >> specifically about >> > interruption though. Let me explain the nuances in a bit >> more detail. >> > >> > With the suggestion we would still fully support if the task >> hasn't yet >> > started that it would be cancellable. >> > >> > However when you cancel a task, there is an >> mayInterruptIfRunning boolean. >> > If mayInterruptIfRunning is true and the task is already >> running it will try >> > to interrupt the thread processing it. This is what this >> email trail is >> > about. We all know that java interruption can be flaky to >> begin with and >> > then adding a remote aspect to it, it becomes even more >> unreliable. >> > >> > So do you guys think we need to support "interrupting" a >> task in progress? >> >> Yes, absolutely! Resources are precious :) >> >> See also the reasoning of our old friend: >> - https://corner.squareup.com/2016/01/query-sniper.html >> >> Take the example of a non-correctly tuned Hibernate Search >> "rebuild >> all indexes" task. On a non-trivial data set it might need to >> run for >> weeks... you need a way to kill it. >> >> >> >> That's the case of our Mass Indexer: one of the motivations of >> making it return a Future was >> the possibility of cancelling tasks, capability offered by the >> Distributed Executor. Could you expand a bit on when you need to interrupt the indexer?

Note that cancel(true) doesn't/didn't do anything if the future is already done (e.g. because the replication timeout expired). So when it comes to handling timeouts, it's always better to do it in the task itself. Cheers Dan >> >> Gustavo >> >> >> Thanks, >> Sanne >> >> > >> > - Will >> > >> > On Wed, Mar 30, 2016 at 12:29 PM Tristan Tarrant >> <ttarrant(a)redhat.com <mailto:ttarrant@redhat.com>> >> > wrote: >> >> >> >> I agree with Sanne, we need cancellable tasks. >> >> >> >> Tristan >> >> >> >> On 30/03/2016 18:19, Sanne Grinovero wrote: >> >> > The term "Interruption" might have been too specific, but >> being able >> >> > to cancel a task seems essential to me. >> >> > >> >> > On 30 March 2016 at 17:04, William Burns >> <mudokonman(a)gmail.com <mailto:mudokonman@gmail.com>> wrote: >> >> >> Recently we have been moving a lot of our methods that >> return Future >> >> >> [1] to >> >> >> CompletableFuture [2]. Unfortunately the latter, >> CompletableFuture, >> >> >> doesn't >> >> >> allow for cancellation of the future, since there is no >> thread tied to >> >> >> it. >> >> >> So I am proposing that our DistributedExecutorService >> [3] no longer >> >> >> allows >> >> >> for interruption of remote threads on a cancellation. >> This way we can >> >> >> have >> >> >> our distributed executor service return >> CompletableFuture instances >> >> >> which do >> >> >> not support interruption. >> >> >> >> >> >> Also to note that DistributedExecutorService extends >> ExecutorService >> >> >> which >> >> >> returns a normal Future which is documented as allowing >> cancellation. >> >> >> In >> >> >> this case I would just document on >> DistributedExecutorService that we >> >> >> don't >> >> >> support interruption anyways. >> >> >> >> >> >> Does anyone require the use of interruptable tasks with >> Distributed >> >> >> Executor? >> >> >> >> >> >> Thanks, >> >> >> >> >> >> - Will >> >> >> >> >> >> [1] >> >> >> >> >> >> >> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html >> >> >> [2] >> >> >> >> >> >> >> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Completabl... >> >> >> [3] >> >> >> >> >> >> >> https://docs.jboss.org/infinispan/8.1/apidocs/org/infinispan/distexec/Dis... >> >> >> >> >> >> _______________________________________________ >> >> >> infinispan-dev mailing list >> >> >> infinispan-dev(a)lists.jboss.org >> <mailto:infinispan-dev@lists.jboss.org> >> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> > _______________________________________________ >> >> > infinispan-dev mailing list >> >> > infinispan-dev(a)lists.jboss.org >> <mailto:infinispan-dev@lists.jboss.org> >> >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> > >> >> >> >> -- >> >> Tristan Tarrant >> >> Infinispan Lead >> >> JBoss, a division of Red Hat >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev(a)lists.jboss.org >> <mailto:infinispan-dev@lists.jboss.org> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev(a)lists.jboss.org >> <mailto:infinispan-dev@lists.jboss.org> >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa <rvansa(a)redhat.com&gt; > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 03/30/2016 12:26 PM, Sanne Grinovero wrote: > On 30 March 2016 at 17:40, William Burns <mudokonman(a)gmail.com&gt; wrote: >> You can still attempt to cancel a task. This email is specifically about >> interruption though. Let me explain the nuances in a bit more detail. >> >> With the suggestion we would still fully support if the task hasn't yet >> started that it would be cancellable. >> >> However when you cancel a task, there is an mayInterruptIfRunning boolean. >> If mayInterruptIfRunning is true and the task is already running it will try >> to interrupt the thread processing it. This is what this email trail is >> about. We all know that java interruption can be flaky to begin with and >> then adding a remote aspect to it, it becomes even more unreliable. >> >> So do you guys think we need to support "interrupting" a task in progress? > > Yes, absolutely! Resources are precious :) > > See also the reasoning of our old friend: > - https://corner.squareup.com/2016/01/query-sniper.html > > Take the example of a non-correctly tuned Hibernate Search "rebuild > all indexes" task. On a non-trivial data set it might need to run for > weeks... you need a way to kill it. Do you want a way to "kill" it, or a way to "interrupt" it? Your reply appeared to equate these, but the terms are actually quite different. Interruption is more of a "hey, if you don't mind could you please stop?", and requires the code to correctly handle it. The vast majority of code isn't written with interruption in mind.

As William mentioned, interruption is not reliable (and there isn't any reliable and safe way to stop a single thread in Java). So the question is whether we want to support "ask it to stop, and it might stop or it might not" functionality.

-Dennis > Thanks, > Sanne > >> >> - Will >> >> On Wed, Mar 30, 2016 at 12:29 PM Tristan Tarrant <ttarrant(a)redhat.com&gt; >> wrote: >>> >>> I agree with Sanne, we need cancellable tasks. >>> >>> Tristan >>> >>> On 30/03/2016 18:19, Sanne Grinovero wrote: >>>> The term "Interruption" might have been too specific, but being able >>>> to cancel a task seems essential to me. >>>> >>>> On 30 March 2016 at 17:04, William Burns <mudokonman(a)gmail.com&gt; wrote: >>>>> Recently we have been moving a lot of our methods that return Future >>>>> [1] to >>>>> CompletableFuture [2]. Unfortunately the latter, CompletableFuture, >>>>> doesn't >>>>> allow for cancellation of the future, since there is no thread tied to >>>>> it. >>>>> So I am proposing that our DistributedExecutorService [3] no longer >>>>> allows >>>>> for interruption of remote threads on a cancellation. This way we can >>>>> have >>>>> our distributed executor service return CompletableFuture instances >>>>> which do >>>>> not support interruption. >>>>> >>>>> Also to note that DistributedExecutorService extends ExecutorService >>>>> which >>>>> returns a normal Future which is documented as allowing cancellation. >>>>> In >>>>> this case I would just document on DistributedExecutorService that we >>>>> don't >>>>> support interruption anyways. >>>>> >>>>> Does anyone require the use of interruptable tasks with Distributed >>>>> Executor? >>>>> >>>>> Thanks, >>>>> >>>>> - Will >>>>> >>>>> [1] >>>>> >>>>> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html >>>>> [2] >>>>> >>>>> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Completabl... >>>>> [3] >>>>> >>>>> https://docs.jboss.org/infinispan/8.1/apidocs/org/infinispan/distexec/Dis... >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>> >>> -- >>> Tristan Tarrant >>> Infinispan Lead >>> JBoss, a division of Red Hat >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev