10:07 a.m.
Vladimir,
Here are some thoughts on your latest iteration (Version 8) of this page:
http://community.jboss.org/wiki/InfinispanDistributedExecutionFramework
There are actually 2 distinct use cases that this feature would touch:
1) pushing a Callable to the node where state is located and pull back a result
2) breaking up a task into Callables and then pushing the Callables out as per (1).
Now only (2) is true map/reduce (and relevant to fork/join) but for many cases, (1) alone
is enough. So whatever API we propose should support both. Simple remote code execution
to leverage data locality where the user doesn't care about breaking up tasks into
subtasks, as well as proper task decomposition.
But we should be clear about these differences - even though on an impl level they are
very closely related. The stuff you have so far handles case (2) pretty well but we
should also expose (1).
Some other feedback:
* I presume distributed task monitoring and annotations are sections that come under
"outside of scope"? They seem to be on the same level so I wasn't sure of
your intentions here.
* Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
* DistributedCallable#preferredExecutionNodes() - do we really want to support this at
this stage? Or is it better to not support arbitrary node selection by end-user code?
Simpler would be to add a DistributedCallable#getRelatedKeys() which returns a Set of keys
which the callable would be expected to touch, so we can decide where to route the task.
Or maybe you want to offer both forms, so that a DistributedCallable could *either*
provide a set of nodes to execute on or a set of keys which it expects to touch.
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
10:36 a.m.
Hey Manik,
On 2010-07-30, at 11:07 AM, Manik Surtani wrote:
Vladimir,
Here are some thoughts on your latest iteration (Version 8) of this page:
http://community.jboss.org/wiki/InfinispanDistributedExecutionFramework
There are actually 2 distinct use cases that this feature would touch:
1) pushing a Callable to the node where state is located and pull back a result
2) breaking up a task into Callables and then pushing the Callables out as per (1).
Now only (2) is true map/reduce (and relevant to fork/join) but for many cases, (1) alone
is enough. So whatever API we propose should support both. Simple remote code execution
to leverage data locality where the user doesn't care about breaking up tasks into
subtasks, as well as proper task decomposition.
Yes, that the intention. Build support for 2) but allow 1). Will add interfaces to define
1. I wanted to get some consensus on how 2) should look like and then focus on 1).
But we should be clear about these differences - even though on an impl level they are
very closely related. The stuff you have so far handles case (2) pretty well but we
should also expose (1).
Will make it clear.
Some other feedback:
* I presume distributed task monitoring and annotations are sections that come under
"outside of scope"? They seem to be on the same level so I wasn't sure of
your intentions here.
Ok will make this distinction more clear.
* Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
The intent of initialized call was to bootstrap task into environment at the master node,
the node invoking the distribution of callables. The purpose of initialize is two fold.
Task implementor can look up input that was given to it in constructor of
DistributedCallable and interpret this input (e.g which keys to use, which particular
cache etc etc) and in turn inform execution environment which cache it would use for data
input so proper ConsistentHash can be passed in preferredExecutionNodes by the
environment. I wanted to isolate task implementor from digging deep into Infinispan
internals to figure out how to fulfill preferredExecutionNodes contract.
preferredExecutionNodes is invoked while task is still at master node before migration to
execution node.
The purpose of mapped is to signal task that it has been migrated to execution node and
that execution is about to begin. Here implementors would actually load the data which is
local to JVM and needed for task computation.
* DistributedCallable#preferredExecutionNodes() - do we really want
to support this at this stage? Or is it better to not support arbitrary node selection by
end-user code?
Task can still do arbitrary node selection in DistributedTask#map(). This goes into
fulfilling objective of 1 from above.
Simpler would be to add a DistributedCallable#getRelatedKeys() which
returns a Set of keys which the callable would be expected to touch, so we can decide
where to route the task. Or maybe you want to offer both forms, so that a
DistributedCallable could *either* provide a set of nodes to execute on or a set of keys
which it expects to touch.
I still like the original approach better :) Maybe it makes more sense with this
elaboration now....
Regards,
Vladimir
10:50 a.m.
On 30 Jul 2010, at 16:36, Vladimir Blagojevic wrote:
> * Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
The intent of initialized call was to bootstrap task into environment at the master node,
the node invoking the distribution of callables. The purpose of initialize is two fold.
Task implementor can look up input that was given to it in constructor of
DistributedCallable and interpret this input (e.g which keys to use, which particular
cache etc etc) and in turn inform execution environment which cache it would use for data
input so proper ConsistentHash can be passed in preferredExecutionNodes by the
environment. I wanted to isolate task implementor from digging deep into Infinispan
internals to figure out how to fulfill preferredExecutionNodes contract.
preferredExecutionNodes is invoked while task is still at master node before migration to
execution node.
Not sure I understand. If you are passing in a reference to the cache the Callable is
executed on (in mapped()) and you are passing in a ConsistentHash in
preferredExecutionNodes(), what is the purpose of initialized()? Why would a Callable
need a reference to the Cache instance at the "master node", as you put it?
The purpose of mapped is to signal task that it has been migrated to
execution node and that execution is about to begin. Here implementors would actually load
the data which is local to JVM and needed for task computation.
Yup, this makes sense.
> * DistributedCallable#preferredExecutionNodes() - do we really
want to support this at this stage? Or is it better to not support arbitrary node
selection by end-user code?
Task can still do arbitrary node selection in DistributedTask#map(). This goes into
fulfilling objective of 1 from above.
Well, like I said this needs to work in a simpler, remote Callable execution case as well
as a Fork/Join task. So in this case we should allow the Callable implementor to either
a) provide a list of preferred nodes, or 2) provide a set of affected keys and let us
decide which nodes to route the task to, based on the keys provided.
> Simpler would be to add a DistributedCallable#getRelatedKeys()
which returns a Set of keys which the callable would be expected to touch, so we can
decide where to route the task. Or maybe you want to offer both forms, so that a
DistributedCallable could *either* provide a set of nodes to execute on or a set of keys
which it expects to touch.
I still like the original approach better :) Maybe it makes more sense with this
elaboration now....
Well, that exposes a ConsistentHash instance to the Callable impl. In most cases, the
Callable wouldn't need to know or care about a ConsistentHash, all it would be
concerned with is the keys it needs. Not the location. We should be able to figure out
location ourselves.
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
11:10 a.m.
On 2010-07-30, at 11:50 AM, Manik Surtani wrote:
On 30 Jul 2010, at 16:36, Vladimir Blagojevic wrote:
>> * Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
>
> The intent of initialized call was to bootstrap task into environment at the master
node, the node invoking the distribution of callables. The purpose of initialize is two
fold. Task implementor can look up input that was given to it in constructor of
DistributedCallable and interpret this input (e.g which keys to use, which particular
cache etc etc) and in turn inform execution environment which cache it would use for data
input so proper ConsistentHash can be passed in preferredExecutionNodes by the
environment. I wanted to isolate task implementor from digging deep into Infinispan
internals to figure out how to fulfill preferredExecutionNodes contract.
preferredExecutionNodes is invoked while task is still at master node before migration to
execution node.
Not sure I understand. If you are passing in a reference to the cache the Callable is
executed on (in mapped()) and you are passing in a ConsistentHash in
preferredExecutionNodes(), what is the purpose of initialized()? Why would a Callable
need a reference to the Cache instance at the "master node", as you put it?
Sure, sure. I complicated it unnecessarily :)
We might need only one callback on master node. But how do we do design it properly to
accommodate the need of the task to do some task input interpretation using interaction
with cache and return list of preferred execution nodes at the same time?
How about just DistributedCallable#getRelatedKeys(EmbbededCacheManager cm)?
1:01 p.m.
On 2010-07-30, at 12:10 PM, Vladimir Blagojevic wrote:
Sure, sure. I complicated it unnecessarily :)
We might need only one callback on master node. But how do we do design it properly to
accommodate the need of the task to do some task input interpretation using interaction
with cache and return list of preferred execution nodes at the same time?
How about just DistributedCallable#getRelatedKeys(EmbbededCacheManager cm)?
Actually we need to know which particular Cache DistributedCallable(s) will use in
addition to keys.
3:47 a.m.
On 30 Jul 2010, at 17:10, Vladimir Blagojevic wrote:
On 2010-07-30, at 11:50 AM, Manik Surtani wrote:
>
> On 30 Jul 2010, at 16:36, Vladimir Blagojevic wrote:
>
>>> * Proposed interfaces - Not sure if I understand the purpose of
DistributedCallable#mapped(). You already assign a cache to the callable in
DistributedCallable#initialized(), right?
>>
>> The intent of initialized call was to bootstrap task into environment at the
master node, the node invoking the distribution of callables. The purpose of initialize is
two fold. Task implementor can look up input that was given to it in constructor of
DistributedCallable and interpret this input (e.g which keys to use, which particular
cache etc etc) and in turn inform execution environment which cache it would use for data
input so proper ConsistentHash can be passed in preferredExecutionNodes by the
environment. I wanted to isolate task implementor from digging deep into Infinispan
internals to figure out how to fulfill preferredExecutionNodes contract.
preferredExecutionNodes is invoked while task is still at master node before migration to
execution node.
>
> Not sure I understand. If you are passing in a reference to the cache the Callable
is executed on (in mapped()) and you are passing in a ConsistentHash in
preferredExecutionNodes(), what is the purpose of initialized()? Why would a Callable
need a reference to the Cache instance at the "master node", as you put it?
Sure, sure. I complicated it unnecessarily :)
We might need only one callback on master node. But how do we do design it properly to
accommodate the need of the task to do some task input interpretation using interaction
with cache and return list of preferred execution nodes at the same time?
This is why I think the task should not care about nodes. :) The task should just be
concerned with data.
How about just
DistributedCallable#getRelatedKeys(EmbbededCacheManager cm)?
I still don't see why you need to pass it the cache manager. Perhaps:
DistributedCallable {
Set<K> getRelatedKeys();
void mappedToNode(Cache<K, V> cache);
}
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
9:31 a.m.
I would love to make it this simple but how does execution environment know which cache to
pass in mappedToNode? Task Implementors somehow have to inform the environment about the
cache they need to retrieve data....
I still don't see why you need to pass it the cache manager. Perhaps:
DistributedCallable {
Set<K> getRelatedKeys();
void mappedToNode(Cache<K, V> cache);
}
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
9:34 a.m.
Simple: the environment first calls getRelatedKeys(), and the impl passes back the keys it
would need. The env then maps the task to a node, based on locating the keys, serializes
and pushes the callable to the assigned node. The assigned node then calls mappedToNode()
so that the impl has a ref to the cache in the local VM. After that, call() is invoked.
On 2 Aug 2010, at 15:31, vblagoje(a)redhat.com wrote:
I would love to make it this simple but how does execution
environment know which cache to pass in mappedToNode? Task Implementors somehow have to
inform the environment about the cache they need to retrieve data....
>
> I still don't see why you need to pass it the cache manager. Perhaps:
>
> DistributedCallable {
>
> Set<K> getRelatedKeys();
>
> void mappedToNode(Cache<K, V> cache);
>
> }
>
>
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
10:24 a.m.
Manik and I agreed on API we need in DistributedCallable:
Map<String, Set<K>> getRelatedKeys();
void mappedToNode(CacheManager cm);
I will update the document accordingly.
On 2010-08-02, at 10:36, Manik Surtani <manik(a)jboss.org> wrote:
Simple: the environment first calls getRelatedKeys(), and the impl
passes back the keys it would need. The env then maps the task to a node, based on
locating the keys, serializes and pushes the callable to the assigned node. The assigned
node then calls mappedToNode() so that the impl has a ref to the cache in the local VM.
After that, call() is invoked.
On 2 Aug 2010, at 15:31, vblagoje(a)redhat.com wrote:
> I would love to make it this simple but how does execution environment know which
cache to pass in mappedToNode? Task Implementors somehow have to inform the environment
about the cache they need to retrieve data....
>
>
>>
>> I still don't see why you need to pass it the cache manager. Perhaps:
>>
>> DistributedCallable {
>>
>> Set<K> getRelatedKeys();
>>
>> void mappedToNode(Cache<K, V> cache);
>>
>> }
>>
>>
>> --
>> Manik Surtani
>> manik(a)jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
5068
days inactive
5071
days old
infinispan-dev@lists.jboss.org
8 comments
3 participants
participants (3)
-
Manik Surtani -
vblagoje＠redhat.com -
Vladimir Blagojevic