[infinispan-issues] [JBoss JIRA] (ISPN-4372) Map/Reduce performance is dependent on cache value size

[Vladimir Blagojevic (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974928#comment-12974928 ] 

Vladimir Blagojevic commented on ISPN-4372:
-------------------------------------------

I am working on another algorithm that uses consumer/producer pattern with bounded queue. Collectors are put in the bounded queue and each threads grabs one, does map (and possibly combine) and returns collector to the queue again. This way we do not have to have all these collector synchronization blocks and copying of collector contents. There is certainly a bit more thread coordination around this queue and more memory is used. Initial testing shows performance equal to current algorithm - at least for local execution. The ongoing work is at https://github.com/vblagoje/infinispan/commits/t_4372 branch

> Map/Reduce performance is dependent on cache value size
> -------------------------------------------------------
>
>                 Key: ISPN-4372
>                 URL: https://issues.jboss.org/browse/ISPN-4372
>             Project: Infinispan
>          Issue Type: Feature Request
>          Components: Distributed Execution and Map/Reduce
>    Affects Versions: 7.0.0.Alpha4
>            Reporter: Alan Field
>            Assignee: Vladimir Blagojevic
>              Labels: performance
>
> Performance testing the Map/Reduce changes has shown that the performance improvements vary based on the size of the values in the cache. [1] Using values from 8kB to 128kB shows a large performance increase over Infinispan 6, but smaller and larger values are the same or slower than Infinispan 6.
> http://blog.infinispan.org/2014/06/mapreduce-performance-improvements.html



--
This message was sent by Atlassian JIRA
(v6.2.3#6260)