[infinispan-issues] [JBoss JIRA] (ISPN-5179) Add distributed execution and map/reduce job statistics

[Vladimir Blagojevic (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036747#comment-13036747 ] 

Vladimir Blagojevic commented on ISPN-5179:
-------------------------------------------

For the first iteration I propose the following levels of statistics information:

- System wide level
- MapReduceTask level

System wide level includes MapReduceTask ids of the running tasks and completed tasks. We could keep a limited history of completed tasks. A sample output for system wide statistics could be:

{code:title=Sample response|borderStyle=solid}
{
"masterNode":"IP address",
"inprogressCount":"2",
"completedCount":"2",
"inprogress":[
     {"id":"actualID1"},
     {"id":"actualID2"}
],
"completed":[
     {"id":"actualID3"},
     {"id":"actualID4"}
]
}
{code}

We could include any other system wide statistic related to map reduce - as the need arise and as we see fit.


MapReduceTask level - given an id of a task in progress or a completed task would return statistics related to that particular task. A sample output could be: 
{code:title=Sample response|borderStyle=solid}
 {
            "runningReduceAttempts" : 1,
            "reduceProgress" : 72.104515,
            "failedReduceAttempts" : 0,
            "mapsRunning" : 0,
            "state" : "RUNNING",
            "reducesRunning" : 1,
            "reducesPending" : 0,
            "mapsCompleted" : 1,
            "startTime" : 1326860720902,
            "id" : "actualID3",
            "runningMapAttempts" : 0,
            "mapsPending" : 0,
            "elapsedTime" : 64432,
            "reducesCompleted" : 0,
            "mapProgress" : 100,
            "failedMapAttempts" : 0,
            "finishTime" : 0
   }
{code}
 




> Add distributed execution and map/reduce job statistics 
> --------------------------------------------------------
>
>                 Key: ISPN-5179
>                 URL: https://issues.jboss.org/browse/ISPN-5179
>             Project: Infinispan
>          Issue Type: Feature Request
>          Components: JMX, reporting and management
>            Reporter: Vladimir Blagojevic
>            Assignee: Vladimir Blagojevic
>             Fix For: 7.2.0.Final
>
>
> We should add DMR/JMX statistics for the running distributed execution jobs as well as map/reduce jobs. The statistics will also include overview/total system statistics of previously executed jobs; we might store statistics of individual executed jobs in some internal cache. However, the primary objective is to calculate and maintain dist.exec and map/reduce job statistics for Infinispan admin console. 



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)