[infinispan-dev] Distributed execution framework

[Manik Surtani]

On 22 Jul 2010, at 14:59, vblagoje at redhat.com wrote:

> 
> 
> On 2010-07-22, at 9:43, Manik Surtani <manik at jboss.org> wrote:
> 
>> Thanks for writing this up dude.  It's nice and clear, pretty much what I had in mind.
>> 
>> A few points.
>> 
>> First, the section on Distributed Tasks Input.  I am guessing what you are trying to model here is access to the actual cache?  Doesn't the Callable in question just need a reference to the Cache?  Not sure I get what you mean by node-local data.  Are you assuming a view of global state that is locally resident on a node in question?  If so, double-counting may still happen if the task ends up on 2 nodes, both of which happen to own an entry.
> 
> Two nodes can own the same entry? We can always us CH to find primary owner, right?!

Well, 2 (or more) nodes can be an owner.  It is better to not assume "primary" versus any other form of owner, as it adds complications during a rehash process.  Not impossible though, and if such order is crucial, it is implicit in any CH algorithm so we could make use of it.  It just needs careful thought from a rehash perspective - e.g., how does this change or affect any code dependent on order if the "primary" were to leave.

> 
>> 
>> Second, failover, perhaps initially we could just ship with a simple policy of throw an exception if a node fails.  And later add more failover and task migration policies.  I'd prefer to start simple and build out more complex features.  :)
> 
> Ok true. Time permitting. Maybe we get some help from the community as well ;)

Yep, precisely.  We need to leverage the fact that we are open source.  :)  Make things pluggable - give people the tools and the means.

Cheers
Manik
--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org