[jbossts-issues] [JBoss JIRA] (JBTM-1453) Reinstate/review the numbers to detect performance regressions

Michael Musgrove (JIRA) issues at jboss.org
Sun Jun 22 19:10:24 EDT 2014

    [ https://issues.jboss.org/browse/JBTM-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978413#comment-12978413 ] 

Michael Musgrove commented on JBTM-1453:

I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.

TODO: I also used Toms job config, JBTM-1453, for obtaining JTS figures using JacORB and SunOrb but the test suite does not store benchmarks for later comparison with previous jobs and failing the build if the variance exceeds the configurable threshold (controlled by the system property "org.jboss.jbossts.qa.Utils.PerformanceVariance"). NB success/failure is tested via org.jboss.jbossts.qa.Utils.PerformanceProfileStore#checkVariance(String performanceName, float operationDuration)

TODO: Tom's extra requirement (RTS and XTS) involves new work. I have some tests for RTS that I can probably integrate into the test suite without too much trouble. I will discuss how to add the XTS requirement with Gytis.

> Reinstate/review the numbers to detect performance regressions
> --------------------------------------------------------------
>                 Key: JBTM-1453
>                 URL: https://issues.jboss.org/browse/JBTM-1453
>             Project: JBoss Transaction Manager
>          Issue Type: Task
>      Security Level: Public(Everyone can see) 
>          Components: Performance Testing
>            Reporter: Tom Jenkinson
>            Assignee: Michael Musgrove
>             Fix For: 5.0.3
>   Original Estimate: 3 days
>  Remaining Estimate: 3 days

This message was sent by Atlassian JIRA

More information about the jbossts-issues mailing list