M/R: Run the combiner concurrently with the mapper -------------------------------------------------- Key: ISPN-4022 URL: https://issues.jboss.org/browse/ISPN-4022 Project: Infinispan Issue Type: Feature Request Components: Core, Distributed Execution and Map/Reduce Affects Versions: 6.0.1.Final Reporter: Dan Berindei Assignee: Vladimir Blagojevic Labels: 630 Fix For: 7.0.0.Alpha3, 7.0.0.Final Because we only run the combiner after we finished the mapping phase, we need to keep all the results of the mapping phase in memory at once. We should split the output of the mapper into chunks and allow the combiner to process chunks while the mapper is still running, relieving some of the memory pressure. Maybe even block the mapper if there are too many chunks in-flight.