[infinispan-issues] [JBoss JIRA] (ISPN-3999) Invoke reducer immediatelly as intermediate VOut values are being collected

[Vladimir Blagojevic (JIRA)]

     [ https://issues.jboss.org/browse/ISPN-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vladimir Blagojevic updated ISPN-3999:
--------------------------------------

    Summary: Invoke reducer immediatelly as intermediate VOut values are being collected  (was: Invoke reducer while intermediate values are being collected)

    
> Invoke reducer immediatelly as intermediate VOut values are being collected
> ---------------------------------------------------------------------------
>
>                 Key: ISPN-3999
>                 URL: https://issues.jboss.org/browse/ISPN-3999
>             Project: Infinispan
>          Issue Type: Enhancement
>            Reporter: Vladimir Blagojevic
>            Assignee: Mircea Markus
>
> Currently in intermediate phase [1] as values for every KOut are being collected/grouped in the cluster we simply add every VOut to the list of values key by KOut. In reducer phase we then send a reducer function that gets invoked on List<VOut> and reduces those values into single VOut. However, this could be suboptimal as we can invoke reducer function as every VOut gets grouped rather than wait for all VOut values to be collected and then invoke reducer. However, reduce function could potentially take a lot of time to execute even for one value thus causing prolonged locking and congestions. We should implement this enhancement and do thorough performance measurements. This feature should be have an API setting in MapReduceTask.
> Sanne Grinovero gets the credit for this idea. 
> [1] http://blog.infinispan.org/2012/07/mapreduce-improvements-in-infinispan.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira