[infinispan-issues] [JBoss JIRA] (ISPN-4318) Infinispan should collect statistics for M/R tasks

[Alan Field (JIRA)]

Alan Field created ISPN-4318:
--------------------------------

             Summary: Infinispan should collect statistics for M/R tasks
                 Key: ISPN-4318
                 URL: https://issues.jboss.org/browse/ISPN-4318
             Project: Infinispan
          Issue Type: Feature Request
          Components: Distributed Execution and Map/Reduce
    Affects Versions: 7.0.0.Alpha4
            Reporter: Alan Field
            Assignee: Dan Berindei


Map/Reduce tasks should collect statistics during the task execution that can be returned to the user to help them determine the optimal settings for the task. Here are some thoughts on useful statistics:

Final status - completed, failed, cancelled, etc.
Duration - either overall, per node, per phase (map, reduce, combine, collate)
Number of nodes participating in the task
Keys in the intermediate cache
Keys in the result map
Node specific statistics:
   Status of node - completed, failed, cancelled, etc.
   Number of keys processed
   Max size of collector

Here are the built in counters that are reported by Hadoop:
https://www.inkling.com/read/hadoop-definitive-guide-tom-white-3rd/chapter-8/counters
   
   



--
This message was sent by Atlassian JIRA
(v6.2.3#6260)