Re: [infinispan-dev] Parallel M/R

On 12/6/2013, 11:40 AM, Mircea Markus wrote: Hi, for those interested in parallel M/R I have uploaded my first proposal that will hopefully, with your input, soon turn into a PR. A couple of things to note: we do parallel iteration using JDK8 ConcurrentHashMap port (see DefaultDataContainer). Doing parallel iteration makes sense when we have to iterate over all keys in data container and check whether each individual key should be traversed (using Mircea's filter). Sometimes it does not make sense to iterate over all keys as input keys are provided to M/R task and input keys represent only a small subset of all keys in the data container. Even in those cases we do parallel execution but in a slightly different approach (again see DefaultDataContainer). Please provide feedback directly on https://github.com/vblagoje/infinispan/tree/t_2284_mm or if you have some general comments here on dev list. Thanks for you feedback, Vladimir