10:07 a.m.
Vladimir,
Here are some thoughts on your latest iteration (Version 8) of this page:
http://community.jboss.org/wiki/InfinispanDistributedExecutionFramework
There are actually 2 distinct use cases that this feature would touch:
1) pushing a Callable to the node where state is located and pull back a result
2) breaking up a task into Callables and then pushing the Callables out as per (1).
Now only (2) is true map/reduce (and relevant to fork/join) but for many cases, (1) alone
is enough. So whatever API we propose should support both. Simple remote code execution
to leverage data locality where the user doesn't care about breaking up tasks into
subtasks, as well as proper task decomposition.
But we should be clear about these differences - even though on an impl level they are
very closely related. The stuff you have so far handles case (2) pretty well but we
should also expose (1).
Some other feedback:
* I presume distributed task monitoring and annotations are sections that come under
"outside of scope"? They seem to be on the same level so I wasn't sure of
your intentions here.
* Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
* DistributedCallable#preferredExecutionNodes() - do we really want to support this at
this stage? Or is it better to not support arbitrary node selection by end-user code?
Simpler would be to add a DistributedCallable#getRelatedKeys() which returns a Set of keys
which the callable would be expected to touch, so we can decide where to route the task.
Or maybe you want to offer both forms, so that a DistributedCallable could *either*
provide a set of nodes to execute on or a set of keys which it expects to touch.
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
10:36 a.m.
Hey Manik,
On 2010-07-30, at 11:07 AM, Manik Surtani wrote:
Yes, that the intention. Build support for 2) but allow 1). Will add interfaces to define
1. I wanted to get some consensus on how 2) should look like and then focus on 1).
Will make it clear.
Ok will make this distinction more clear.
* Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
The intent of initialized call was to bootstrap task into environment at the master node,
the node invoking the distribution of callables. The purpose of initialize is two fold.
Task implementor can look up input that was given to it in constructor of
DistributedCallable and interpret this input (e.g which keys to use, which particular
cache etc etc) and in turn inform execution environment which cache it would use for data
input so proper ConsistentHash can be passed in preferredExecutionNodes by the
environment. I wanted to isolate task implementor from digging deep into Infinispan
internals to figure out how to fulfill preferredExecutionNodes contract.
preferredExecutionNodes is invoked while task is still at master node before migration to
execution node.
The purpose of mapped is to signal task that it has been migrated to execution node and
that execution is about to begin. Here implementors would actually load the data which is
local to JVM and needed for task computation.
* DistributedCallable#preferredExecutionNodes() - do we really want
to support this at this stage? Or is it better to not support arbitrary node selection by
end-user code?
Task can still do arbitrary node selection in DistributedTask#map(). This goes into
fulfilling objective of 1 from above.
I still like the original approach better :) Maybe it makes more sense with this
elaboration now....
Regards,
Vladimir
10:50 a.m.
On 30 Jul 2010, at 16:36, Vladimir Blagojevic wrote:
Not sure I understand. If you are passing in a reference to the cache the Callable is
executed on (in mapped()) and you are passing in a ConsistentHash in
preferredExecutionNodes(), what is the purpose of initialized()? Why would a Callable
need a reference to the Cache instance at the "master node", as you put it?
The purpose of mapped is to signal task that it has been migrated to
execution node and that execution is about to begin. Here implementors would actually load
the data which is local to JVM and needed for task computation.
Yup, this makes sense.
Well, like I said this needs to work in a simpler, remote Callable execution case as well
as a Fork/Join task. So in this case we should allow the Callable implementor to either
a) provide a list of preferred nodes, or 2) provide a set of affected keys and let us
decide which nodes to route the task to, based on the keys provided.
Well, that exposes a ConsistentHash instance to the Callable impl. In most cases, the
Callable wouldn't need to know or care about a ConsistentHash, all it would be
concerned with is the keys it needs. Not the location. We should be able to figure out
location ourselves.
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
11:10 a.m.
On 2010-07-30, at 11:50 AM, Manik Surtani wrote:
Sure, sure. I complicated it unnecessarily :)
We might need only one callback on master node. But how do we do design it properly to
accommodate the need of the task to do some task input interpretation using interaction
with cache and return list of preferred execution nodes at the same time?
How about just DistributedCallable#getRelatedKeys(EmbbededCacheManager cm)?
3:47 a.m.
On 30 Jul 2010, at 17:10, Vladimir Blagojevic wrote:
This is why I think the task should not care about nodes. :) The task should just be
concerned with data.
How about just
DistributedCallable#getRelatedKeys(EmbbededCacheManager cm)?
I still don't see why you need to pass it the cache manager. Perhaps:
DistributedCallable {
Set<K> getRelatedKeys();
void mappedToNode(Cache<K, V> cache);
}
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
9:34 a.m.
Simple: the environment first calls getRelatedKeys(), and the impl passes back the keys it
would need. The env then maps the task to a node, based on locating the keys, serializes
and pushes the callable to the assigned node. The assigned node then calls mappedToNode()
so that the impl has a ref to the cache in the local VM. After that, call() is invoked.
On 2 Aug 2010, at 15:31, vblagoje(a)redhat.com wrote:
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
5355
days inactive
5358
days old
infinispan-dev@lists.jboss.org
8 comments
3 participants
participants (3)
-
Manik Surtani -
vblagoje＠redhat.com -
Vladimir Blagojevic