[infinispan-dev] Distributed execution framework - API proposal(s)

[Manik Surtani]

I agree with Sanne.  A top-level Scala API may be nice, but definitely lower in prio when compared to a more traditional Java-centric API using Java paradigms.  The latter is still what most folks will use for some time to come.


On 5 Jan 2011, at 13:11, Sanne Grinovero wrote:

> While I agree on the importance of a nice API, I'd polish the Java one first.
> This article from May 2008 is already showing off a pretty nice example:
> http://www.infoq.com/news/2008/03/fork_join
> 
> IMHO even the Java-coded version is very nice, and I don't foresee
> people writing hundreds of F/J functions, so clarity is far more
> important than brevity and syntax coolness.
> I'm not against a nice Scala API, always nice to have, but the Java
> one should be made understandable first. (And sorry it might be my
> lack of field experience, but I didn't understand how I should code a
> simple exercise with the current proposals, while the F/J example is
> self-speaking coolness).
> 
> Sanne
> 
> 2011/1/5 Galder Zamarreño <galder at redhat.com>:
>> 
>> On Jan 5, 2011, at 11:30 AM, Galder Zamarreño wrote:
>> 
>>> Something else to add here. Earlier today I was looking at GridGain and the functional way to define map/reduce functions and seemed quite appealing, particularly cos they had managed to do it for Java as well:
>>> "Native Java & Scala Support" section in http://www.gridgain.com/product.html
>>> 
>>> Having contracted the Scala virus last year, this way of coding functions looks more appealing (to me) than the way they did in 2.0 which is fairly close to what is currently being proposed.
>>> 
>>> Maybe something not for the first version, but something definitely worth keeping in mind.
>> 
>> A few years this might have been a niche, but since Scala and Java 7 (later this year), I see this way of coding becoming more and more ubiquitous, so maybe we should do this right from the start? I could certainly give a hand with this part.
>> 
>>> 
>>> On Dec 27, 2010, at 5:25 PM, Vladimir Blagojevic wrote:
>>> 
>>>> Hey,
>>>> 
>>>> I spent the last week working on concrete API proposals for distributed
>>>> execution framework. I believe that we are close to finalize the
>>>> proposal and your input and feedback is important now! Here are the main
>>>> ideas where I think we made progress since we last talked.
>>>> 
>>>> 
>>>> Access to multiple caches during task execution
>>>> 
>>>> While we have agreed to allow access to multiple caches during task
>>>> execution including this logic into task API complicates it greatly. The
>>>> compromise I found is to focus all API on to a one specific cache but
>>>> allow access to other caches through DistributedTaskContext API. The
>>>> focus on one specific cache and its input keys will allows us to
>>>> properly CH map task units across Infinispan cluster and will cover most
>>>> of the use cases. DistributedTaskContext can also easily be mapped to a
>>>> single cache. See DistributedTask and DistributedTaskContext for more
>>>> details.
>>>> 
>>>> 
>>>> DistributedTask and DistributedCallable
>>>> 
>>>> I found it useful to separate task characteristics in general and actual
>>>> work/computation details. Therefore the main task characteristics are
>>>> specified through DistributedTask API and details of actual task
>>>> computation are specified through DistributedCallable API.
>>>> DistributedTask specifies coarse task details, the failover policy, the
>>>> task splitting policy, cancellation policy and so on while in
>>>> DistributedCallable API implementers focus on actual details of a
>>>> computation/work unit.
>>>> 
>>>> 
>>>> I have updated the original document [1] to reflect API update. You can
>>>> see the actual proposal in git here [2] and I have also included the
>>>> variation of this approach [3] that separates map and reduce task phases
>>>> with separate interfaces and removes DistributedCallable interaface. I
>>>> have also kept Trustin's ideas in another proposal [4] since I would
>>>> like to include them as well if possible.
>>>> 
>>>> Regards,
>>>> Vladimir
>>>> 
>>>> 
>>>> [1] http://community.jboss.org/wiki/InfinispanDistributedExecutionFramework
>>>> [2] https://github.com/vblagoje/infinispan/tree/t_ispn-39_master_prop1
>>>> [3] https://github.com/vblagoje/infinispan/tree/t_ispn-39_master_prop2
>>>> [4] https://github.com/vblagoje/infinispan/tree/t_ispn-39_master_prop3
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> --
>>> Galder Zamarreño
>>> Sr. Software Engineer
>>> Infinispan, JBoss Cache
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org