Re: [infinispan-dev] New draft for distributed execution framework

Hi Trustin Thanks for this. A few comments: * 1st section, by node affinity, I presume you mean data locality? * 2nd section: data processing tasks would be sent out to specific nodes where required data is available. Not to every node. * Do we want to support this for replicated mode? Typically, replicated clusters will be small and won't really benefit from task execution. Maybe we add in support for replication later, if there is adequate demand? It shouldn't change the API, etc. * +1 re: multi-cores on each node. I presume this would rely on an Executor set up on each node for this purpose? And a user would have the opportunity to configure this executor (e.g., based on the type of hardware in use)? * Could you define where each of these tasks are executed? E.g., in a grid of {A, B, C, D} and you invoke your example on A, the local tasks happen on any node which contains the data it needs, while the global task happens on A? Surely in many cases it makes sense to "reduce" before pushing results (final or intermediate) on the network? * Non-Java language support - this should be out of scope for now, as it adds a layer of complexity we don't need. We will certainly add support for map/reduce over Hot Rod in future, perhaps using JSON/JavaScript, but not for now. :) * Client/Server development: again, out of scope for now. See above. Once we have map/reduce over Hot Rod, client/server would be trivial. Cheers Manik On 30 Nov 2010, at 09:12, Trustin Lee wrote: -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org