Lets take the 2 API sets (M/R and distributed exec) separately:

* Why does MapReduceTask extend Set? It probably shouldn't even extend DistributedRemoteTask, since some methods such as execute(Callable) doesn't make sense for MapReduceTask. Perhaps a common super-interface? That alone should simplify the generics...

* Why does Mapper.map() return a Map? [1] Shouldn't it return a single transformation for a given K and V?

* Reducer.reduce()'s return type should not necessarily be the same type as the mapped transformation. Also, does the reducer really need the key? Surely just the transformed version of each K/V pair.

* Similarly, Collator.add() should just need the address and the reduced result from each node (each node would only produce 1 result!)

* Why do you have execute() and executeAsync() with no params? What do these methods do?

* Not sure what DistributedTaskContext is all about. What does read() and write() do? Is this meant to be a "layer" in front of the cache in question? Why not just pass in a ref to the cache?

* What does Factory do?

[1] How many times can I say "map" in one sentence? :)

On 10 Jan 2011, at 19:28, Vladimir Blagojevic wrote:

On 11-01-05 3:39 PM, Manik Surtani wrote:

Awesome! I love this fluent API proposal!

Cool. Then if this is the case lets stick with the fluent API - I think it would make life much easier for developers.

--

Manik Surtani

Proposal Manik and I talked about is here https://github.com/vblagoje/infinispan/commit/59b0c3cb78bce70fdcd49d2d36f68de110944fc6

I have slightly changed it to better match map/reduce paradigm described here http://labs.google.com/papers/mapreduce.html
Therefore, we now have both map/reduce and a more simple "execute this task unit on remote nodes and return results" proposal.
Users might not like number of type parameters in MapReduceTask. I wanted to be as precise as possible to begin with and we can discuss how and if to reduce them.
Example for map/reduce is WordCountExample.java. Have a look.

I put a proposal in distexec package. I wanted to have parent package to branch from. We need to accommodate simple execution model, map/reduce, various execution policies and so on. They could not fit all in mapreduce package and its children. Hence distexec package. If you think you have better name - speak up.

Regards,
Vladimir

_______________________________________________
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Manik Surtani

manik@jboss.org

twitter.com/maniksurtani

Lead, Infinispan

http://www.infinispan.org