[infinispan-dev] Distributed executors and Future(s) they return

[Sanne Grinovero]

On 27 February 2014 18:40, Dan Berindei <dan.berindei at gmail.com> wrote:
>
>
>
> On Thu, Feb 27, 2014 at 8:13 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
>>
>> On 27 February 2014 16:58, Mircea Markus <mmarkus at redhat.com> wrote:
>> >
>> > On Feb 27, 2014, at 3:28 PM, Vladimir Blagojevic <vblagoje at redhat.com>
>> > wrote:
>> >
>> >> Hmm very good points Sanne. Yeah I think we can have a contract that
>> >> returns an Address were task was executed.
>> >>
>> >>
>> >> Cheers,
>> >> Vladimir
>> >> On 2/26/2014, 4:25 PM, Sanne Grinovero wrote:
>> >>> I'm a bit skeptical.
>> >>> It might sound a sensible request currently, but if you do so you
>> >>> inherently "promise" that tasks are going to be executed on a specific
>> >>> server; AFAIK we promise execution on data locality,
>> >
>> > We allow execution to be bound on a specific address:
>> > http://goo.gl/H5qTJZ
>>
>> I know but I think that smells :)
>> Stuff like _Address_ should be an implementation detail. Maybe one day
>> you'll see why and we'll deprecate it ;-)
>>
>> > I see your point with data locality vs. specific server.
>> >
>> >
>> >>> but maintaining a
>> >>> good level of flexibility you can evolve your system to smarter load
>> >>> balancing of tasks, failover operations, etc..
>> >>> If you expose execution details, you won't be able to develop any of
>> >>> that in future.
>> >>>
>> >>> To make an example from the database world - seems the analogy is
>> >>> common these days - it's like you run a SELECT statement but want to
>> >>> pick which CPU core is going to be used. That would be really odd, as
>> >>> you would take away the option from the scheduler to make an effective
>> >>> choice.
>> >>> Still, this approach might be desirable for a database which doesn't
>> >>> do any smart scheduling.
>> >>>
>> >>> Some of these concerns might be mitigated if you return the Address of
>> >>> where the task *was* executed, after it's done. I still don't think it
>> >>> should be of user's interest but at least you would be able to
>> >>> implement rescheduling or failover policies in future.
>> >
>> > We already have failure policies in place, but the user only needs to
>> > audit the failure, not to failover. If users are interested on knowing the
>> > failures, another way of doing it is the current future, in the Future.get
>> > to throw a custom exception (subclass of ExecutionException) containing as
>> > information where the execution failed.
>>
>> Right, but the question is if the user really wants to know the
>> intermediate failures? I suspect that if someone asks for this, he's
>> actually wishing to implement his own failower policy & monitoring.
>> >From the point of view of someone running a database query, I think
>> the user would love to ignore issues altogether, but the real world
>> forces him to at least consider that the whole operation might fail.
>> Sending him specific notifications or exceptions of something that was
>> succesfull but was actually run on a different resource set than what
>> was originally planned is I'd say an exotic request.
>
>
> I don't think the user was after the address of the "real" executing node, I
> believe he just wanted a way to map each Future to the target address doing
> a submitEverywhere(task).
>
>>
>>
>> I like the idea of providing additional information in a Future
>> subtype, but I don't think you should throw it on a get() operation.
>> You could simply add getters to the FutureExtended to retrieve like an
>> execution plan history, a trace of intermediate failures, etc.
>>
>
> That sounds good, but we shouldn't limit that to just the result of one
> distributed task execution. We could take the opportunity to return
> something other than List<Future> from submitEverywhere(task) as well, doing
> a foreach to get all the results is a bit tedious. And even if we'd like
> users to treat the results from different nodes as interchangeable,
> sometimes they're not, so a way of getting the result from one particular
> node would also be useful.

+1

>
>
>>
>> Sanne
>>
>> >
>> >>>
>> >>> Sanne
>> >>>
>> >>>
>> >>> On 26 February 2014 19:31, Vladimir Blagojevic <vblagoje at redhat.com>
>> >>> wrote:
>> >>>> Hey,
>> >>>>
>> >>>> There is an interesting request from community to include an Address
>> >>>> along with a Future returned for a subtask being executed [1].
>> >>>>
>> >>>> I think it makes sense what this user wants. We might create Future
>> >>>> sub interface that has getAddress method and we can return an object
>> >>>> implementing that interface instead of plain Future. In some new major
>> >>>> release we can officially change the signature of these
>> >>>> DistributedExecutorService methods to return i.e TargetedFuture - it would
>> >>>> not break existing clients. Maybe even make TargetedFuture extend
>> >>>> NotifyingFuture.
>> >>>>
>> >>>> Any thoughts?
>> >>>>
>> >>>> Vladimir
>> >>>>
>> >>>> [1] https://community.jboss.org/thread/237442
>> >>>>
>> >>>> _______________________________________________
>> >>>> infinispan-dev mailing list
>> >>>> infinispan-dev at lists.jboss.org
>> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>> _______________________________________________
>> >>> infinispan-dev mailing list
>> >>> infinispan-dev at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>> > Cheers,
>> > --
>> > Mircea Markus
>> > Infinispan lead (www.infinispan.org)
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev