[infinispan-dev] Distributed execution framework - API proposal(s)

[Manik Surtani]

On 12 Jan 2011, at 12:17, Vladimir Blagojevic wrote:

> On 11-01-11 2:42 PM, Manik Surtani wrote:
>> How about this pseudocode:
>> 
>> On each node:
>>   mapped = list()
>>   for entry in cache.entries:
>>      t = mapper.map(entry.key, entry.value)
>>      mapped.add(t)
>> 
>>   r = null
>>   for t in mapped:
>>      r = reducer.reduce(t, r)
>> 
>>   return r to calling node
>> 
>> On calling node:
>>   reduced_results = invoke map reduce task on all nodes, retrieve 
>> map{address: result}
>>   for r in reduced_results.entries:
>>      remote_address = r.key
>>      remote_reduced_result = r.value
>>      collator.add(remote_address, remote_reduced_result)
>> 
>>   return collator.collate()
>> 
> 
> Interesting Manik! Where did you come up with this model? The examples I 
> tried to write using this model (WordCount etc) would work yet your 
> model is definitely different than the original proposed map/reduce!

I just made it up. :) Keep in mind that the original Google paper was designed to be very generic in terms of use case as well as implementation.

Cheers
Manik

--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org