[infinispan-issues] [JBoss JIRA] (ISPN-4318) Infinispan should collect statistics for M/R tasks

[William Burns (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179240#comment-13179240 ] 

William Burns commented on ISPN-4318:
-------------------------------------

We may want to port this over to distributed streams if possible.

> Infinispan should collect statistics for M/R tasks
> --------------------------------------------------
>
>                 Key: ISPN-4318
>                 URL: https://issues.jboss.org/browse/ISPN-4318
>             Project: Infinispan
>          Issue Type: Feature Request
>          Components: Distributed Execution and Map/Reduce
>    Affects Versions: 7.0.0.Alpha4
>            Reporter: Alan Field
>
> Map/Reduce tasks should collect statistics during the task execution that can be returned to the user to help them determine the optimal settings for the task. Here are some thoughts on useful statistics:
> Final status - completed, failed, cancelled, etc.
> Duration - either overall, per node, per phase (map, reduce, combine, collate)
> Number of nodes participating in the task
> Keys in the intermediate cache
> Keys in the result map
> Node specific statistics:
>    Status of node - completed, failed, cancelled, etc.
>    Number of keys processed
>    Max size of collector
> Here are the built in counters that are reported by Hadoop:
> https://www.inkling.com/read/hadoop-definitive-guide-tom-white-3rd/chapter-8/counters
>    
>    



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)