[jbossts-issues] [JBoss JIRA] (JBTM-1453) Reinstate/review the numbers to detect performance regressions

Michael Musgrove (JIRA) issues at jboss.org
Wed Jun 25 10:17:24 EDT 2014


    [ https://issues.jboss.org/browse/JBTM-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978413#comment-12978413 ] 

Michael Musgrove edited comment on JBTM-1453 at 6/25/14 10:15 AM:
------------------------------------------------------------------

I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
 
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.

TODO: Tom's extra XTS requirement involves new work which I will discuss with Gytis.


was (Author: mmusgrov):
I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
 
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.

TODO: Tom's extra requirement (RTS and XTS) involves new work. I have some tests for RTS that I can probably integrate into the test suite without too much trouble. I will discuss how to add the XTS requirement with Gytis.

TODO: The job includes axes for JTS testing with JacORB and SunOrb but those tests currently do not report perf regressions  via org.jboss.jbossts.qa.Utils.PerformanceProfileStore#checkVariance(String performanceName, float operationDuration)) so I have commented those axes for now.

> Reinstate/review the numbers to detect performance regressions
> --------------------------------------------------------------
>
>                 Key: JBTM-1453
>                 URL: https://issues.jboss.org/browse/JBTM-1453
>             Project: JBoss Transaction Manager
>          Issue Type: Task
>      Security Level: Public(Everyone can see) 
>          Components: Performance Testing
>            Reporter: Tom Jenkinson
>            Assignee: Michael Musgrove
>             Fix For: 5.0.3
>
>   Original Estimate: 3 days
>  Remaining Estimate: 3 days
>




--
This message was sent by Atlassian JIRA
(v6.2.6#6264)


More information about the jbossts-issues mailing list