Re: [infinispan-dev] Distributed execution framework
On 22 Jul 2010, at 14:59, vblagoje(a)redhat.com wrote:
On 2010-07-22, at 9:43, Manik Surtani <manik(a)jboss.org> wrote:
> Thanks for writing this up dude. It's nice and clear, pretty much what I had in
mind.
>
> A few points.
>
> First, the section on Distributed Tasks Input. I am guessing what you are trying to
model here is access to the actual cache? Doesn't the Callable in question just need
a reference to the Cache? Not sure I get what you mean by node-local data. Are you
assuming a view of global state that is locally resident on a node in question? If so,
double-counting may still happen if the task ends up on 2 nodes, both of which happen to
own an entry.
Two nodes can own the same entry? We can always us CH to find primary owner, right?!
Well, 2 (or more) nodes can be an owner. It is better to not assume "primary"
versus any other form of owner, as it adds complications during a rehash process. Not
impossible though, and if such order is crucial, it is implicit in any CH algorithm so we
could make use of it. It just needs careful thought from a rehash perspective - e.g., how
does this change or affect any code dependent on order if the "primary" were to
leave.
>
> Second, failover, perhaps initially we could just ship with a simple policy of throw
an exception if a node fails. And later add more failover and task migration policies.
I'd prefer to start simple and build out more complex features. :)
Ok true. Time permitting. Maybe we get some help from the community as well ;)
Yep, precisely. We need to leverage the fact that we are open source. :) Make things
pluggable - give people the tools and the means.
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org