[JBoss JIRA] (JBTM-1453) Reinstate/review the numbers to detect performance regressions
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1453?page=com.atlassian.jira.plugin.... ]
Michael Musgrove edited comment on JBTM-1453 at 6/25/14 10:15 AM:
------------------------------------------------------------------
I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.
TODO: Tom's extra XTS requirement involves new work which I will discuss with Gytis.
was (Author: mmusgrov):
I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.
TODO: Tom's extra requirement (RTS and XTS) involves new work. I have some tests for RTS that I can probably integrate into the test suite without too much trouble. I will discuss how to add the XTS requirement with Gytis.
TODO: The job includes axes for JTS testing with JacORB and SunOrb but those tests currently do not report perf regressions via org.jboss.jbossts.qa.Utils.PerformanceProfileStore#checkVariance(String performanceName, float operationDuration)) so I have commented those axes for now.
> Reinstate/review the numbers to detect performance regressions
> --------------------------------------------------------------
>
> Key: JBTM-1453
> URL: https://issues.jboss.org/browse/JBTM-1453
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Performance Testing
> Reporter: Tom Jenkinson
> Assignee: Michael Musgrove
> Fix For: 5.0.3
>
> Original Estimate: 3 days
> Remaining Estimate: 3 days
>
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)