[jbossts-issues] [JBoss JIRA] (JBTM-3137) Too much variance in the performance PR checker

[Mayank Kunwar (Jira)]

    [ https://issues.redhat.com/browse/JBTM-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988692#comment-13988692 ] 

Mayank Kunwar commented on JBTM-3137:
-------------------------------------

I did testing in my local system which has 8 CPU cores, so I observed different result in CI system, since that system has 24 cores.

> Too much variance in the performance PR checker
> -----------------------------------------------
>
>                 Key: JBTM-3137
>                 URL: https://issues.redhat.com/browse/JBTM-3137
>             Project: JBoss Transaction Manager
>          Issue Type: Bug
>          Components: Performance Testing
>    Affects Versions: 5.9.5.Final
>            Reporter: Michael Musgrove
>            Assignee: Mayank Kunwar
>            Priority: Major
>             Fix For: 5.next
>
>
> The PERF axis of the PR checker job runs some tests to validate that the PR does not introduce any regressions but the axis is not useful since there is too much variance in the runs. For example PR https://github.com/jbosstm/narayana/pull/1437 ran 3 times each producing wildly different results.



--
This message was sent by Atlassian Jira
(v7.13.8#713008)