<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Resending, since for some reason I can't find this message in the archives and wonder if it has been delivered. :/<div><br></div><div>Vladimir, this is what I meant by a more JDK-like API for distributed executors.</div><div><br></div><div>Cheers</div><div>Manik</div><div><br></div><div><br></div><div><div><br><div>Begin forwarded message:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>From: </b></span><span style="font-family:'Helvetica'; font-size:medium;">Manik Surtani <<a href="mailto:manik@jboss.org">manik@jboss.org</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>Date: </b></span><span style="font-family:'Helvetica'; font-size:medium;">11 January 2011 14:45:11 GMT<br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>To: </b></span><span style="font-family:'Helvetica'; font-size:medium;">infinispan -Dev List <<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>Subject: </b></span><span style="font-family:'Helvetica'; font-size:medium;"><b>Re: [infinispan-dev] Distributed execution framework - API proposal(s)</b><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>Reply-To: </b></span><span style="font-family:'Helvetica'; font-size:medium;">infinispan -Dev List <<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a>><br></span></div><br><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">And for the distributed executor API, how's this as an alternative (suggested by Eduardo earlier)?<div><br></div><div>DistributedExecutorService extends ExecutorService [1], adds overloaded methods of submit(Callable) with submit(Callable, K...). In keeping with the ExecutorService contract, submit() will ensure the Callable is executed just once, on any one node. We could add a submitEverywhere(Callable) and submitEverywhere(Callable, K...) to mean execution on _all_ nodes.</div><div><br></div><div>Also ships with a DistributedCallable which extends Callable, adds setCache() and setEmbeddedCacheManager() setters. If the callable is a DistributedCallable, then these setters are called before Callable.call(); otherwise just do Callable.call().</div><div><br></div><div>WDYT?</div><div><br></div><div>[1] <a href="http://download.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html">http://download.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html</a></div><div><br><div><div>On 11 Jan 2011, at 14:18, Vladimir Blagojevic wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On 11-01-11 10:28 AM, Manik Surtani wrote:<br><blockquote type="cite">Hi there.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Lets take the 2 API sets (M/R and distributed exec) separately:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">M/R<br></blockquote><blockquote type="cite">* Why does MapReduceTask extend Set? It probably shouldn't even <br></blockquote><blockquote type="cite">extend DistributedRemoteTask, since some methods such as <br></blockquote><blockquote type="cite">execute(Callable) doesn't make sense for MapReduceTask. Perhaps a <br></blockquote><blockquote type="cite">common super-interface? That alone should simplify the generics...<br></blockquote><br>Leftover that I did not catch. I meant Map<K1,K2> instead of <br>Set<Map.Entry<K2, V2>>; it does not extend Set it just constraints one <br>of the type parameters :-)<br><br>Completely agree on common parent that has the common setters for <br>various task aspects.<br><br><blockquote type="cite">* Why does Mapper.map() return a Map? [1] Shouldn't it return a single <br></blockquote><blockquote type="cite">transformation for a given K and V?<br></blockquote><br>Yes, I thought so too but how would we then match intermediate keys for <br>reduce phase. Our plumbing has to find all the same <br>transformed/intermediate keys returned from a map phase and then invoke <br>the reduce phase.<br><br><blockquote type="cite">* Reducer.reduce()'s return type should not necessarily be the same <br></blockquote><blockquote type="cite">type as the mapped transformation. Also, does the reducer really need <br></blockquote><blockquote type="cite">the key? Surely just the transformed version of each K/V pair.<br></blockquote><br>Yeah, when we solve the above this should fall into place.<br><br><blockquote type="cite">* Similarly, Collator.add() should just need the address and the <br></blockquote><blockquote type="cite">reduced result from each node (each node would only produce 1 result!)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">DistExec<br></blockquote><blockquote type="cite">* Why do you have execute() and executeAsync() with no params? What <br></blockquote><blockquote type="cite">do these methods do?<br></blockquote><br>In case user provided Factory for DistributedCallable. We have to handle <br>cases where simple no parameter constructor for callables is not <br>sufficient - hence factory. Now that I think about it maybe adding K... <br>input as a parameter to call function of DistributedCallable makes more <br>sense, or no? WDYT?<br><br><blockquote type="cite">* Not sure what DistributedTaskContext is all about. What does read() <br></blockquote><blockquote type="cite">and write() do? Is this meant to be a "layer" in front of the cache <br></blockquote><blockquote type="cite">in question? Why not just pass in a ref to the cache?<br></blockquote><br>True.<br><br><blockquote type="cite">* What does Factory do?<br></blockquote><br>Creates DistributedCallables. See above.<br><br><blockquote type="cite"><br></blockquote><blockquote type="cite">Almost there! :-)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers<br></blockquote><blockquote type="cite">Manik<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">[1] How many times can I say "map" in one sentence? :)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote>Thanks Manik. Almost there :-)<br>_______________________________________________<br>infinispan-dev mailing list<br><a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br><a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br></div></blockquote></div><br><div>
<div><div>--</div><div>Manik Surtani</div><div><a href="mailto:manik@jboss.org">manik@jboss.org</a></div><div><a href="http://twitter.com/maniksurtani">twitter.com/maniksurtani</a></div><div><br></div><div>Lead, Infinispan</div><div><a href="http://www.infinispan.org/">http://www.infinispan.org</a></div><div><br></div></div><br class="Apple-interchange-newline">
</div>
<br></div></div>_______________________________________________<br>infinispan-dev mailing list<br><a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>https://lists.jboss.org/mailman/listinfo/infinispan-dev</blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><div>--</div><div>Manik Surtani</div><div><a href="mailto:manik@jboss.org">manik@jboss.org</a></div><div><a href="http://twitter.com/maniksurtani">twitter.com/maniksurtani</a></div><div><br></div><div>Lead, Infinispan</div><div><a href="http://www.infinispan.org">http://www.infinispan.org</a></div><div><br></div></div></span><br class="Apple-interchange-newline">
</div>
<br></div></body></html>