[infinispan-dev] Parallel M/R

[Galder Zamarreño]

Looks good, I made a comment replying to one of Mircea's comments.

Cheers,

On Dec 12, 2013, at 6:57 PM, Vladimir Blagojevic <vblagoje at redhat.com> wrote:

> On 12/6/2013, 11:40 AM, Mircea Markus wrote:
>> Hmm I think you could leverage the parallel iteration from the EquivalentConcurrentHashMapV8 there instead of writing it yourself ;)
>> 
> Hi, for those interested in parallel M/R I have uploaded my first 
> proposal that will hopefully, with your input, soon turn into a PR. A 
> couple of things to note: we do parallel iteration using JDK8 
> ConcurrentHashMap port (see DefaultDataContainer). Doing parallel 
> iteration makes sense when we have to iterate over all keys in data 
> container and check whether each individual key should be traversed 
> (using Mircea's filter). Sometimes it does not make sense to iterate 
> over all keys as input keys are provided to M/R task and input keys 
> represent only a small subset of all keys in the data container. Even in 
> those cases we do parallel execution but in a slightly different 
> approach (again see DefaultDataContainer).
> 
> Please provide feedback directly on 
> https://github.com/vblagoje/infinispan/tree/t_2284_mm or if you have 
> some general comments here on dev list.
> 
> Thanks for you feedback,
> Vladimir
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org