[infinispan-dev] Map/Reduce enhancements with ISPN-2037

[Galder Zamarreño]

Thanks, I've made my comments. @Dan, can you have a look too?

When this is in, we should consider doing another alpha.

On Jul 18, 2012, at 10:06 PM, Vladimir Blagojevic wrote:

> Right, here it is https://github.com/infinispan/infinispan/pull/1218
> On 12-07-18 12:32 PM, Galder Zamarreño wrote:
>> Hey Vladimir,
>> 
>> Since you have multiple commits for us to comment on, why not send a pull req for that branch?
>> 
>> Much easier from a review POV
>> 
>> Cheers,
>> 
>> On Jul 13, 2012, at 8:13 PM, Vladimir Blagojevic wrote:
>> 
>>> Guys,
>>> 
>>> I believe I finished ISPN-2037 that Sanne needed for his queries.
>>> However, adding ISPN-2037 commit to current master involves injection of
>>> CacheLoaderManager into MapReduceCommand. That solution is pretty ugly
>>> and we are switching soon to a new map/reduce implementation anyway.
>>> Going in that direction is a waste IMHO.
>>> 
>>> I already talked to Mircea and Manik about using a Manager-like service
>>> much like RecoveryManager or EvictionManager to manage incoming
>>> map/reduce invocations on Infinispan node. MapReduceManager is per cache
>>> component. It turned out to be an excellent fit! I converted our current
>>> solution where map is done in distributed fashion while reduction is
>>> done on master task node to this new approach. I made it a default just
>>> as it is now. Then I added all the previous work on new map/reduce
>>> reviewed by Dan and Galder and adjusted it to this new Manager paradigm.
>>> Users can choose if they want reduce part done in distributed approach
>>> simply by using a different MapReduceTask constructor. And finally I
>>> added ISPN-2037 as a separate commit on top of this rework.
>>> 
>>> This proposal is at
>>> https://github.com/vblagoje/infinispan/commits/t_mr_final
>>> 
>>> I would like to hear your thoughts and comments before proceeding forward.
>>> 
>>> Regards,
>>> Vladimir
>>> 
>>> 
> 

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache