[infinispan-dev] Map/Reduce enhancements with ISPN-2037

Vladimir Blagojevic vblagoje at redhat.com
Fri Jul 13 13:13:09 EDT 2012


Guys,

I believe I finished ISPN-2037 that Sanne needed for his queries. 
However, adding ISPN-2037 commit to current master involves injection of 
CacheLoaderManager into MapReduceCommand. That solution is pretty ugly 
and we are switching soon to a new map/reduce implementation anyway. 
Going in that direction is a waste IMHO.

I already talked to Mircea and Manik about using a Manager-like service 
much like RecoveryManager or EvictionManager to manage incoming 
map/reduce invocations on Infinispan node. MapReduceManager is per cache 
component. It turned out to be an excellent fit! I converted our current 
solution where map is done in distributed fashion while reduction is 
done on master task node to this new approach. I made it a default just 
as it is now. Then I added all the previous work on new map/reduce 
reviewed by Dan and Galder and adjusted it to this new Manager paradigm. 
Users can choose if they want reduce part done in distributed approach 
simply by using a different MapReduceTask constructor. And finally I 
added ISPN-2037 as a separate commit on top of this rework.

This proposal is at 
https://github.com/vblagoje/infinispan/commits/t_mr_final

I would like to hear your thoughts and comments before proceeding forward.

Regards,
Vladimir










More information about the infinispan-dev mailing list