12:42 a.m.
Hi folks,
I posted the new draft for the distributed execution framework. It's
written based on the discussion between Vladimir and me. I initially
thought about publishing the API as a git branch, but I guess it's a
good idea to have a couple rounds before doing that. I tried to be as
specific as possible, but please feel free to let me know what could be
improved, what is missing, and so on.
Cheers,
Trustin
3:02 a.m.
Where is the draft? http://community.jboss.org/docs/DOC-15622 shows as last modified date
last August...
On Nov 30, 2010, at 7:42 AM, Trustin Lee wrote:
Hi folks,
I posted the new draft for the distributed execution framework. It's
written based on the discussion between Vladimir and me. I initially
thought about publishing the API as a git branch, but I guess it's a
good idea to have a couple rounds before doing that. I tried to be as
specific as possible, but please feel free to let me know what could be
improved, what is missing, and so on.
Cheers,
Trustin
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
3:04 a.m.
Umm, where is the draft? :-)
On 30 Nov 2010, at 06:42, Trustin Lee wrote:
Hi folks,
I posted the new draft for the distributed execution framework. It's
written based on the discussion between Vladimir and me. I initially
thought about publishing the API as a git branch, but I guess it's a
good idea to have a couple rounds before doing that. I tried to be as
specific as possible, but please feel free to let me know what could be
improved, what is missing, and so on.
Cheers,
Trustin
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
3:12 a.m.
Ouch, here's the URL:
http://community.jboss.org/wiki/DistributedDataStreamProcessingFrameworkI...
On 11/30/2010 03:42 PM, Trustin Lee wrote:
Hi folks,
I posted the new draft for the distributed execution framework. It's
written based on the discussion between Vladimir and me. I initially
thought about publishing the API as a git branch, but I guess it's a
good idea to have a couple rounds before doing that. I tried to be as
specific as possible, but please feel free to let me know what could be
improved, what is missing, and so on.
Cheers,
Trustin
4:11 a.m.
Hi Trustin
Thanks for this. A few comments:
* 1st section, by node affinity, I presume you mean data locality?
* 2nd section: data processing tasks would be sent out to specific nodes where required
data is available. Not to every node.
* Do we want to support this for replicated mode? Typically, replicated clusters will be
small and won't really benefit from task execution. Maybe we add in support for
replication later, if there is adequate demand? It shouldn't change the API, etc.
* +1 re: multi-cores on each node. I presume this would rely on an Executor set up on
each node for this purpose? And a user would have the opportunity to configure this
executor (e.g., based on the type of hardware in use)?
* Could you define where each of these tasks are executed? E.g., in a grid of {A, B, C,
D} and you invoke your example on A, the local tasks happen on any node which contains the
data it needs, while the global task happens on A? Surely in many cases it makes sense to
"reduce" before pushing results (final or intermediate) on the network?
* Non-Java language support - this should be out of scope for now, as it adds a layer of
complexity we don't need. We will certainly add support for map/reduce over Hot Rod
in future, perhaps using JSON/JavaScript, but not for now. :)
* Client/Server development: again, out of scope for now. See above. Once we have
map/reduce over Hot Rod, client/server would be trivial.
Cheers
Manik
On 30 Nov 2010, at 09:12, Trustin Lee wrote:
Ouch, here's the URL:
http://community.jboss.org/wiki/DistributedDataStreamProcessingFrameworkI...
On 11/30/2010 03:42 PM, Trustin Lee wrote:
> Hi folks,
>
> I posted the new draft for the distributed execution framework. It's
> written based on the discussion between Vladimir and me. I initially
> thought about publishing the API as a git branch, but I guess it's a
> good idea to have a couple rounds before doing that. I tried to be as
> specific as possible, but please feel free to let me know what could be
> improved, what is missing, and so on.
>
> Cheers,
> Trustin
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
6:56 a.m.
On 11/30/2010 07:11 PM, Manik Surtani wrote:
Hi Trustin
Thanks for this. A few comments:
* 1st section, by node affinity, I presume you mean data locality?
Yes. Let me fix it.
* 2nd section: data processing tasks would be sent out to specific
nodes where required data is available. Not to every node.
You're right again. :-)
* Do we want to support this for replicated mode? Typically,
replicated clusters will be small and won't really benefit from task execution. Maybe
we add in support for replication later, if there is adequate demand? It shouldn't
change the API, etc.
For example, a user might want to run a cluster of 100 nodes with 2
replicas just like Google File System does. Why is such cluster
supposed to be small? Could you elaborate?
* +1 re: multi-cores on each node. I presume this would rely on an
Executor set up on each node for this purpose? And a user would have the opportunity to
configure this executor (e.g., based on the type of hardware in use)?
Yes, that's what I intended.
* Could you define where each of these tasks are executed? E.g., in
a grid of {A, B, C, D} and you invoke your example on A, the local tasks happen on any
node which contains the data it needs, while the global task happens on A? Surely in many
cases it makes sense to "reduce" before pushing results (final or intermediate)
on the network?
Each node runs the first two sub-tasks and then one node is chosen to
run the last one. Timeline will look like this:
A: Task1 .. Task2 ... Task3
B: Task1 .. Task2 /
C: Task1 .. Task2 /
D: Task1 .. Task2 /
Because Task1 and 2 are executed in the same node, they could be done
via a simple invocation chaining. The transition from Task2 to Task3
will involve network communication. In the example, the output is only
one entry, so it will be kept in memory until the node A's Task3 picks
them up. If Task2 generates a lot of entries, each node will hold them
in memory and block the task execution on ctx.write() until the Task3
picks them up.
One potential problem is when Node A fails while running Task3. Because
the intermediary entries are loaded into memory and lost as soon as they
are consumed, there's no way to re-retrieve the intermediary output and
the entire task needs to be re-run. A user can simply split the task to
two parts to store the intermediary output in the cache.
* Non-Java language support - this should be out of scope for now, as
it adds a layer of complexity we don't need. We will certainly add support for
map/reduce over Hot Rod in future, perhaps using JSON/JavaScript, but not for now. :)
* Client/Server development: again, out of scope for now. See above. Once we have
map/reduce over Hot Rod, client/server would be trivial.
Agreed. I didn't mean to implement in 5.0 unless there is good deal of
demand. :-)
More feed back appreciated in advanced, folks! :-)
--
Trustin Lee, http://gleamynode.net/
10:38 a.m.
On 30 Nov 2010, at 12:56, 이희승 (Trustin Lee) wrote:
On 11/30/2010 07:11 PM, Manik Surtani wrote:
<SNIP />
> * Do we want to support this for replicated mode? Typically,
replicated clusters will be small and won't really benefit from task execution. Maybe
we add in support for replication later, if there is adequate demand? It shouldn't
change the API, etc.
For example, a user might want to run a cluster of 100 nodes with 2
replicas just like Google File System does. Why is such cluster
supposed to be small? Could you elaborate?
Ah, in Infinispan Replication refers to REPL mode, which means every node maintains a
replica. E.g., if you have 100 nodes, you have 100 copies of everything.
Distribution refers to using a Consistent Hash algorithm to maintain a fixed number of
copies (e.g., 2) regardless of how big a cluster is (e.g., 100 nodes). I see the F/J
framework being useful here.
http://docs.jboss.org/infinispan/4.2/apidocs/org/infinispan/config/Config...
> * Could you define where each of these tasks are executed? E.g.,
in a grid of {A, B, C, D} and you invoke your example on A, the local tasks happen on any
node which contains the data it needs, while the global task happens on A? Surely in many
cases it makes sense to "reduce" before pushing results (final or intermediate)
on the network?
Each node runs the first two sub-tasks and then one node is chosen to
run the last one. Timeline will look like this:
A: Task1 .. Task2 ... Task3
B: Task1 .. Task2 /
C: Task1 .. Task2 /
D: Task1 .. Task2 /
Right, but this is specific to your example. How would one specify what sort of
characteristics they need from each task?
Because Task1 and 2 are executed in the same node, they could be done
via a simple invocation chaining. The transition from Task2 to Task3
will involve network communication. In the example, the output is only
one entry, so it will be kept in memory until the node A's Task3 picks
them up. If Task2 generates a lot of entries, each node will hold them
in memory and block the task execution on ctx.write() until the Task3
picks them up.
One potential problem is when Node A fails while running Task3. Because
the intermediary entries are loaded into memory and lost as soon as they
are consumed, there's no way to re-retrieve the intermediary output and
the entire task needs to be re-run. A user can simply split the task to
two parts to store the intermediary output in the cache.
> * Non-Java language support - this should be out of scope for now, as it adds a layer
of complexity we don't need. We will certainly add support for map/reduce over Hot
Rod in future, perhaps using JSON/JavaScript, but not for now. :)
> * Client/Server development: again, out of scope for now. See above. Once we have
map/reduce over Hot Rod, client/server would be trivial.
Agreed. I didn't mean to implement in 5.0 unless there is good deal of
demand. :-)
More feed back appreciated in advanced, folks! :-)
--
Trustin Lee, http://gleamynode.net/
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
5:07 a.m.
On 30 Nov 2010, at 18:46, Vladimir Blagojevic wrote:
On 10-11-30 1:38 PM, Manik Surtani wrote:
>
> Right, but this is specific to your example. How would one specify
> what sort of characteristics they need from each task?
In terms of task failover, error handling, cancellation etc? Is that
what you are referring to?
No, in terms of which tasks are executed 'locally' and which tasks are executed
'remotely'.
E.g., there are concievably 3 types of tasks:
1) Tasks that are data intensive, that should be executed on a remote node, close to the
actual data
2) Aggregation of results of these data intensive tasks, which should happen on remote
nodes as well, before transmitting any results back to the caller
3) Aggregation of several results on the caller.
Now each of these tasks can be chained - I think there definitely is value in that.
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
7:05 a.m.
On 10-12-01 8:07 AM, Manik Surtani wrote:
No, in terms of which tasks are executed 'locally' and which
tasks are executed 'remotely'.
E.g., there are concievably 3 types of tasks:
1) Tasks that are data intensive, that should be executed on a remote node, close to the
actual data
2) Aggregation of results of these data intensive tasks, which should happen on remote
nodes as well, before transmitting any results back to the caller
3) Aggregation of several results on the caller.
Now each of these tasks can be chained - I think there definitely is value in that.
Their specifics are determined as they are added to "coordinated task",
I am sure we can tune this to requirements we need.
|CacheManager cm = ...;
CoordinatedTask task = cm.newCoordinatedTask("finalOutcomeCacheName");
cm.addLocalTask(*new* GridFileQuery("my_gridfs"));// Fetch
cm.addLocalTask(*new* GridFileSizeCounter());// Map
cm.addGlobalTask(*new* IntegerSummarizer("my_gridfs_size");// Reduce|
8:39 a.m.
Ok, so addLocalTask() will cause the task to run on the remote node, and addGlobalTask()
causes the task to run on the local node? :-)
On 1 Dec 2010, at 13:05, Vladimir Blagojevic wrote:
On 10-12-01 8:07 AM, Manik Surtani wrote:
>
> No, in terms of which tasks are executed 'locally' and which tasks are
executed 'remotely'.
>
> E.g., there are concievably 3 types of tasks:
>
> 1) Tasks that are data intensive, that should be executed on a remote node, close to
the actual data
> 2) Aggregation of results of these data intensive tasks, which should happen on
remote nodes as well, before transmitting any results back to the caller
> 3) Aggregation of several results on the caller.
>
> Now each of these tasks can be chained - I think there definitely is value in that.
Their specifics are determined as they are added to "coordinated task", I am
sure we can tune this to requirements we need.
CacheManager cm = ...;
CoordinatedTask task = cm.newCoordinatedTask("finalOutcomeCacheName");
cm.addLocalTask(new GridFileQuery("my_gridfs")); // Fetch
cm.addLocalTask(new GridFileSizeCounter()); // Map
cm.addGlobalTask(new IntegerSummarizer("my_gridfs_size"); // Reduce
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
8:46 a.m.
:-) And I presume users will be able to tune specifics with additional
parameters to addLocal/GlobalTask etc etc. It should be really cool!
On 10-12-01 11:39 AM, Manik Surtani wrote:
Ok, so addLocalTask() will cause the task to run on the remote node,
and addGlobalTask() causes the task to run on the local node? :-)
8:54 a.m.
It would be cool, but I'd prefer less ambiguous names. ;)
On 1 Dec 2010, at 14:46, Vladimir Blagojevic wrote:
:-) And I presume users will be able to tune specifics with
additional
parameters to addLocal/GlobalTask etc etc. It should be really cool!
On 10-12-01 11:39 AM, Manik Surtani wrote:
> Ok, so addLocalTask() will cause the task to run on the remote node,
> and addGlobalTask() causes the task to run on the local node? :-)
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
4920
days inactive
4921
days old
infinispan-dev@lists.jboss.org
12 comments
5 participants
participants (5)
-
"이희승 (Trustin Lee)" -
Galder Zamarreño -
Manik Surtani -
Trustin Lee
-
Vladimir Blagojevic