[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris commented on JBTM-2170:
---------------------------------------
[~mmusgrov], yeah I get that. But weren't we building jacorb only by default before?
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Michael Musgrove commented on JBTM-2170:
----------------------------------------
The logic says if IBM_ORB then disable idlj and jacorb otherwise enable idlj (and implicitly enable jacorb since jacorb-disabled is not set).
So by default we always build both idlj and jacorb.
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris commented on JBTM-2170:
---------------------------------------
just noticed that in jacoco config there is an exclusion
{code}
<exclude name="**/Idlj/**"/>
{code}
However, it should be from lower case. Changing this should fix the issue.
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris commented on JBTM-2170:
---------------------------------------
Yes, just checking why does it build idlj in the first place. Wasn't it excluded from the default build at first? It seems that narayana.sh not enables it if IBM orb is not enabled:
{code}
if [ $IBM_ORB = 1 ]; then
ORBARG="-Dibmorb-enabled -Djacorb-disabled"
${JAVA_HOME}/bin/java -version 2>&1 | grep IBM
[ $? = 0 ] || fatal "You must use the IBM jdk to build with ibmorb"
else
ORBARG="-Didlj-enabled"
fi
{code}
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Michael Musgrove commented on JBTM-2170:
----------------------------------------
There is an excludes section in the jacoco-report target where we exclude problem classes.
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris commented on JBTM-2170:
---------------------------------------
Problem is with ildj and jacorb modules having the class with the same name: com/arjuna/ArjunaOTS/ActionControlHelper
{code}
BUILD FAILED
/home/gytis/workspace/narayana/narayana/qa/run-tests.xml:540: Error while creating report
at org.jacoco.ant.ReportTask.execute(ReportTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.Main.runBuild(Main.java:851)
at org.apache.tools.ant.Main.startAnt(Main.java:235)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Caused by: java.io.IOException: Error while analyzing class ArjunaJTS/idl/jacorb/target/classes/com/arjuna/ArjunaOTS/ActionControlHelper.class.
at org.jacoco.core.analysis.Analyzer.analyzerError(Analyzer.java:139)
at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:133)
at org.jacoco.core.analysis.Analyzer.analyzeAll(Analyzer.java:164)
at org.jacoco.ant.ReportTask.createBundle(ReportTask.java:570)
at org.jacoco.ant.ReportTask.createReport(ReportTask.java:545)
at org.jacoco.ant.ReportTask.execute(ReportTask.java:492)
... 16 more
Caused by: java.lang.IllegalStateException: Can't add different class with same name: com/arjuna/ArjunaOTS/ActionControlHelper
at org.jacoco.core.analysis.CoverageBuilder.visitCoverage(CoverageBuilder.java:89)
at org.jacoco.core.analysis.Analyzer$1.visitEnd(Analyzer.java:81)
at org.jacoco.asm.ClassVisitor.visitEnd(Unknown Source)
at org.jacoco.core.internal.flow.ClassProbesAdapter.visitEnd(ClassProbesAdapter.java:135)
at org.jacoco.asm.ClassReader.accept(Unknown Source)
at org.jacoco.asm.ClassReader.accept(Unknown Source)
at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:96)
at org.jacoco.core.analysis.Analyzer.analyzeClass(Analyzer.java:131)
... 20 more
{code}
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-2170) Jacoco failed to create QA code coverage report
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-2170?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris reassigned JBTM-2170:
-------------------------------------
Assignee: Gytis Trikleris (was: Michael Musgrove)
> Jacoco failed to create QA code coverage report
> -----------------------------------------------
>
> Key: JBTM-2170
> URL: https://issues.jboss.org/browse/JBTM-2170
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Gytis Trikleris
> Assignee: Gytis Trikleris
> Fix For: 5.0.3
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/narayana-codeCoverage/118
> {code}
> BUILD SUCCESSFUL
> Total time: 510 minutes 46 seconds
> generating test coverage report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 8 seconds
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> has-testoutput-output:
> testoutput.zip:
> [echo] archiving testoutput to testoutput-jacorb.zip
> [zip] Building zip: /home/hudson/workspace/narayana-codeCoverage/qa/testoutput-jacorb.zip
> BUILD SUCCESSFUL
> Total time: 9 seconds
> + rm -rf /home/hudson/.m2/repository/org/jboss/narayana
> + cd qa
> + ant -f run-tests.xml jacoco-report
> Buildfile: /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml
> jacoco-report:
> [jacoco:report] Loading execution data file /home/hudson/workspace/narayana-codeCoverage/qa/testoutput/jacoco-qa.exec
> BUILD FAILED
> /home/hudson/workspace/narayana-codeCoverage/qa/run-tests.xml:540: Error while creating report
> Total time: 9 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Sending e-mails to: gtrikler(a)redhat.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-1453) Reinstate/review the numbers to detect performance regressions
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1453?page=com.atlassian.jira.plugin.... ]
Michael Musgrove edited comment on JBTM-1453 at 6/22/14 7:35 PM:
-----------------------------------------------------------------
I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.
TODO: Tom's extra requirement (RTS and XTS) involves new work. I have some tests for RTS that I can probably integrate into the test suite without too much trouble. I will discuss how to add the XTS requirement with Gytis.
TODO: The job includes axes for JTS testing with JacORB and SunOrb but those tests currently do not report perf regressions via org.jboss.jbossts.qa.Utils.PerformanceProfileStore#checkVariance(String performanceName, float operationDuration)) so I have commented those axes for now.
was (Author: mmusgrov):
I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.
TODO: I also used Toms job config, JBTM-1453, for obtaining JTS figures using JacORB and SunOrb but the test suite does not store benchmarks for later comparison with previous jobs and failing the build if the variance exceeds the configurable threshold (controlled by the system property "org.jboss.jbossts.qa.Utils.PerformanceVariance"). NB success/failure is tested via org.jboss.jbossts.qa.Utils.PerformanceProfileStore#checkVariance(String performanceName, float operationDuration)
TODO: Tom's extra requirement (RTS and XTS) involves new work. I have some tests for RTS that I can probably integrate into the test suite without too much trouble. I will discuss how to add the XTS requirement with Gytis.
> Reinstate/review the numbers to detect performance regressions
> --------------------------------------------------------------
>
> Key: JBTM-1453
> URL: https://issues.jboss.org/browse/JBTM-1453
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Performance Testing
> Reporter: Tom Jenkinson
> Assignee: Michael Musgrove
> Fix For: 5.0.3
>
> Original Estimate: 3 days
> Remaining Estimate: 3 days
>
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months
[JBoss JIRA] (JBTM-1453) Reinstate/review the numbers to detect performance regressions
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1453?page=com.atlassian.jira.plugin.... ]
Michael Musgrove commented on JBTM-1453:
----------------------------------------
I've fixed the "original requirement as stated in the JIRA" and set up a regular job for detecting regressions: view/Narayana+BlackTie/job/narayana-performance.
The job is tied to haverstraw and runs as part of the QA test suite. A test passes if it completes within a 10% variance from the previous successful run.
TODO: I also used Toms job config, JBTM-1453, for obtaining JTS figures using JacORB and SunOrb but the test suite does not store benchmarks for later comparison with previous jobs and failing the build if the variance exceeds the configurable threshold (controlled by the system property "org.jboss.jbossts.qa.Utils.PerformanceVariance"). NB success/failure is tested via org.jboss.jbossts.qa.Utils.PerformanceProfileStore#checkVariance(String performanceName, float operationDuration)
TODO: Tom's extra requirement (RTS and XTS) involves new work. I have some tests for RTS that I can probably integrate into the test suite without too much trouble. I will discuss how to add the XTS requirement with Gytis.
> Reinstate/review the numbers to detect performance regressions
> --------------------------------------------------------------
>
> Key: JBTM-1453
> URL: https://issues.jboss.org/browse/JBTM-1453
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Performance Testing
> Reporter: Tom Jenkinson
> Assignee: Michael Musgrove
> Fix For: 5.0.3
>
> Original Estimate: 3 days
> Remaining Estimate: 3 days
>
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 10 months