January 2020 - jbossts-issues - Jboss List Archives

[JBoss JIRA] (JBTM-3235) Performance regression tests should consult the "Score Error" before reporting regressions

by Mayank Kunwar (Jira)

[ https://issues.redhat.com/browse/JBTM-3235?page=com.atlassian.jira.plugin... ] Mayank Kunwar updated JBTM-3235: -------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done > Performance regression tests should consult the "Score Error" before reporting regressions > ------------------------------------------------------------------------------------------ > > Key: JBTM-3235 > URL: https://issues.redhat.com/browse/JBTM-3235 > Project: JBoss Transaction Manager > Issue Type: Task > Components: Performance Testing > Affects Versions: 5.10.1.Final > Reporter: Michael Musgrove > Assignee: Mayank Kunwar > Priority: Optional > Fix For: 5.next > > > We perform [microbenchmark performance testing of JTA|https://github.com/jbosstm/performance/tree/master/narayana/ArjunaJTA...] and we have Jenkins CI jobs (btny-pulls-performance and narayana-performance-version-comparison) that do regression testing against different versions of the product. > These tests use [JMH|https://openjdk.java.net/projects/code-tools/jmh/] to perform the test runs. The results of each run are reported as a CSV file in the following example format: > {code} > "Version","Benchmark","Mode","Threads","Samples","Score","Score Error (99.9%)","Unit" > 5.5.32.Final,"io.narayana.perf.product.NarayanaComparison.test","thrpt",240,15,217253.976024,8720.877211,"ops/s" > 5.9.0.Final,"io.narayana.perf.product.NarayanaComparison.test","thrpt",240,15,216552.983213,6586.482104,"ops/s" > {code} > This example shows a comparison between versions 5.5.32.Final and 5.9.0.Final. The data shows that > we measured using a mode called "thrpt" (ie transactions per second or TPS) and the Score shows the TPS was 217253 vers > us 216552. There is also a Score Error which I understand indicates the upper and lower error bars for the measurement. > None of our jobs take account of the error bars when reporting regressions. What we need to do is look at the Score Error to see if the runs are within tolerance before reporting a regression. > An example of how to manually run a regression test is as follows: > {code} > git clone https://github.com/jbosstm/performance # clone the performance git repo > cd performance > export JMHARGS="-t 240 -r 30 -f 3 -wi 5 -i 5" # tune the test using JMH config params > # and then do a performance run: > export version=5.5.32.Final # set the target narayana version > ./build.sh -f narayana/ArjunaJTA/jta/pom.xml clean install -DskipTests -Dnarayana.version=$version > java -classpath narayana/ArjunaJTA/jta/target/classes/ -Dcom.arjuna.ats.arjuna.common.propertiesFile=jbossts-properties.xml -jar narayana/ArjunaJTA/jta/target/benchmarks.jar io.narayana.perf.product.NarayanaComparison.* $JMHARGS -rf csv -rff io.narayana.perf.product.NarayanaComparison-$version.csv > {code} > The results are in the named csv file. > Then do the same for the second version -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3246) Support MP transaction context propagation for async calls for CDI

by Ondrej Chaloupka (Jira)

[ https://issues.redhat.com/browse/JBTM-3246?page=com.atlassian.jira.plugin... ] Issue was automatically transitioned when Ondrej Chaloupka created pull request #1558 in GitHub ----------------------------------------------------------------------------------------------- Status: Pull Request Sent (was: Open) > Support MP transaction context propagation for async calls for CDI > ------------------------------------------------------------------ > > Key: JBTM-3246 > URL: https://issues.redhat.com/browse/JBTM-3246 > Project: JBoss Transaction Manager > Issue Type: Enhancement > Components: JTA > Affects Versions: 5.10.3.Final > Reporter: Ondrej Chaloupka > Assignee: Ondrej Chaloupka > Priority: Major > > For full integration of Narayana CDI handling to MicroProfile context propagation it needed to support context propagation for asynchronous calls. > Currently asynchronous call is handled just synchronously. That means the transaction is finished just at time when CDI interceptor finishes with the business method. While there could be some unfinished async call running aside of the main method. The async then may work with already finished transactions which lead to an error. > Here we aim to integrate with MicroProfile Smallrye implementation. The approach will be adapted from Quarkus. The point is to delay finishing of the transaction up to time the async method is finished as well. > The Quarkus code for this here: > https://github.com/quarkusio/quarkus/pull/2321/files#diff-9fae5ceb58cc73d... -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3235) Performance regression tests should consult the "Score Error" before reporting regressions

by Anonymous (Jira)

[ https://issues.redhat.com/browse/JBTM-3235?page=com.atlassian.jira.plugin... ] Issue was automatically transitioned when mayankkunwar created pull request #63 in GitHub ----------------------------------------------------------------------------------------- Status: Pull Request Sent (was: Open) > Performance regression tests should consult the "Score Error" before reporting regressions > ------------------------------------------------------------------------------------------ > > Key: JBTM-3235 > URL: https://issues.redhat.com/browse/JBTM-3235 > Project: JBoss Transaction Manager > Issue Type: Task > Components: Performance Testing > Affects Versions: 5.10.1.Final > Reporter: Michael Musgrove > Assignee: Mayank Kunwar > Priority: Optional > Fix For: 5.next > > > We perform [microbenchmark performance testing of JTA|https://github.com/jbosstm/performance/tree/master/narayana/ArjunaJTA...] and we have Jenkins CI jobs (btny-pulls-performance and narayana-performance-version-comparison) that do regression testing against different versions of the product. > These tests use [JMH|https://openjdk.java.net/projects/code-tools/jmh/] to perform the test runs. The results of each run are reported as a CSV file in the following example format: > {code} > "Version","Benchmark","Mode","Threads","Samples","Score","Score Error (99.9%)","Unit" > 5.5.32.Final,"io.narayana.perf.product.NarayanaComparison.test","thrpt",240,15,217253.976024,8720.877211,"ops/s" > 5.9.0.Final,"io.narayana.perf.product.NarayanaComparison.test","thrpt",240,15,216552.983213,6586.482104,"ops/s" > {code} > This example shows a comparison between versions 5.5.32.Final and 5.9.0.Final. The data shows that > we measured using a mode called "thrpt" (ie transactions per second or TPS) and the Score shows the TPS was 217253 vers > us 216552. There is also a Score Error which I understand indicates the upper and lower error bars for the measurement. > None of our jobs take account of the error bars when reporting regressions. What we need to do is look at the Score Error to see if the runs are within tolerance before reporting a regression. > An example of how to manually run a regression test is as follows: > {code} > git clone https://github.com/jbosstm/performance # clone the performance git repo > cd performance > export JMHARGS="-t 240 -r 30 -f 3 -wi 5 -i 5" # tune the test using JMH config params > # and then do a performance run: > export version=5.5.32.Final # set the target narayana version > ./build.sh -f narayana/ArjunaJTA/jta/pom.xml clean install -DskipTests -Dnarayana.version=$version > java -classpath narayana/ArjunaJTA/jta/target/classes/ -Dcom.arjuna.ats.arjuna.common.propertiesFile=jbossts-properties.xml -jar narayana/ArjunaJTA/jta/target/benchmarks.jar io.narayana.perf.product.NarayanaComparison.* $JMHARGS -rf csv -rff io.narayana.perf.product.NarayanaComparison-$version.csv > {code} > The results are in the named csv file. > Then do the same for the second version -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3246) Support MP transaction context propagation for async calls for CDI

by Ondrej Chaloupka (Jira)

Ondrej Chaloupka created JBTM-3246: -------------------------------------- Summary: Support MP transaction context propagation for async calls for CDI Key: JBTM-3246 URL: https://issues.redhat.com/browse/JBTM-3246 Project: JBoss Transaction Manager Issue Type: Enhancement Components: JTA Affects Versions: 5.10.3.Final Reporter: Ondrej Chaloupka Assignee: Ondrej Chaloupka For full integration of Narayana CDI handling to MicroProfile context propagation it needed to support context propagation for asynchronous calls. Currently asynchronous call is handled just synchronously. That means the transaction is finished just at time when CDI interceptor finishes with the business method. While there could be some unfinished async call running aside of the main method. The async then may work with already finished transactions which lead to an error. Here we aim to integrate with MicroProfile Smallrye implementation. The approach will be adapted from Quarkus. The point is to delay finishing of the transaction up to time the async method is finished as well. The Quarkus code for this here: https://github.com/quarkusio/quarkus/pull/2321/files#diff-9fae5ceb58cc73d... -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3235) Performance regression tests should consult the "Score Error" before reporting regressions

by Michael Musgrove (Jira)

[ https://issues.redhat.com/browse/JBTM-3235?page=com.atlassian.jira.plugin... ] Michael Musgrove reassigned JBTM-3235: -------------------------------------- Assignee: Mayank Kunwar (was: Michael Musgrove) > Performance regression tests should consult the "Score Error" before reporting regressions > ------------------------------------------------------------------------------------------ > > Key: JBTM-3235 > URL: https://issues.redhat.com/browse/JBTM-3235 > Project: JBoss Transaction Manager > Issue Type: Task > Components: Performance Testing > Affects Versions: 5.10.1.Final > Reporter: Michael Musgrove > Assignee: Mayank Kunwar > Priority: Optional > Fix For: 5.next > > > We perform [microbenchmark performance testing of JTA|https://github.com/jbosstm/performance/tree/master/narayana/ArjunaJTA...] and we have Jenkins CI jobs (btny-pulls-performance and narayana-performance-version-comparison) that do regression testing against different versions of the product. > These tests use [JMH|https://openjdk.java.net/projects/code-tools/jmh/] to perform the test runs. The results of each run are reported as a CSV file in the following example format: > {code} > "Version","Benchmark","Mode","Threads","Samples","Score","Score Error (99.9%)","Unit" > 5.5.32.Final,"io.narayana.perf.product.NarayanaComparison.test","thrpt",240,15,217253.976024,8720.877211,"ops/s" > 5.9.0.Final,"io.narayana.perf.product.NarayanaComparison.test","thrpt",240,15,216552.983213,6586.482104,"ops/s" > {code} > This example shows a comparison between versions 5.5.32.Final and 5.9.0.Final. The data shows that > we measured using a mode called "thrpt" (ie transactions per second or TPS) and the Score shows the TPS was 217253 vers > us 216552. There is also a Score Error which I understand indicates the upper and lower error bars for the measurement. > None of our jobs take account of the error bars when reporting regressions. What we need to do is look at the Score Error to see if the runs are within tolerance before reporting a regression. > An example of how to manually run a regression test is as follows: > {code} > git clone https://github.com/jbosstm/performance # clone the performance git repo > cd performance > export JMHARGS="-t 240 -r 30 -f 3 -wi 5 -i 5" # tune the test using JMH config params > # and then do a performance run: > export version=5.5.32.Final # set the target narayana version > ./build.sh -f narayana/ArjunaJTA/jta/pom.xml clean install -DskipTests -Dnarayana.version=$version > java -classpath narayana/ArjunaJTA/jta/target/classes/ -Dcom.arjuna.ats.arjuna.common.propertiesFile=jbossts-properties.xml -jar narayana/ArjunaJTA/jta/target/benchmarks.jar io.narayana.perf.product.NarayanaComparison.* $JMHARGS -rf csv -rff io.narayana.perf.product.NarayanaComparison-$version.csv > {code} > The results are in the named csv file. > Then do the same for the second version -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3239) Failing AfterLRA participant calls are not repeated with recovery module

by Michael Musgrove (Jira)

[ https://issues.redhat.com/browse/JBTM-3239?page=com.atlassian.jira.plugin... ] Michael Musgrove reassigned JBTM-3239: -------------------------------------- Assignee: Michael Musgrove (was: Martin Stefanko) > Failing AfterLRA participant calls are not repeated with recovery module > ------------------------------------------------------------------------ > > Key: JBTM-3239 > URL: https://issues.redhat.com/browse/JBTM-3239 > Project: JBoss Transaction Manager > Issue Type: Bug > Components: LRA > Affects Versions: 5.10.1.Final > Reporter: Martin Stefanko > Assignee: Michael Musgrove > Priority: Major > > The afterLRA calls are currently delivered only twice when the LRA is finished (twice because of JBTM-3163). When these two calls fail the afterLRA call is never repeated again. However, the specification states that afterLRA must be repeated until the 200 status code is returned. > The main reason for this issue is the handling of afterLRA listeners in the Transaction object which means that the LRA enters the end phase (Closed/Cancelled) then the afterLRA listeners are called (twice because of above-mentioned issue) but if these calls fail there is nothing telling the transaction object to repeat the calls on the recovery. -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3245) Check the behaviour when LRA enlistment fails due to coordinator unavailability

by Michael Musgrove (Jira)

Michael Musgrove created JBTM-3245: -------------------------------------- Summary: Check the behaviour when LRA enlistment fails due to coordinator unavailability Key: JBTM-3245 URL: https://issues.redhat.com/browse/JBTM-3245 Project: JBoss Transaction Manager Issue Type: Task Components: LRA Affects Versions: 5.10.3.Final Reporter: Michael Musgrove Assignee: Michael Musgrove Fix For: 5.next There is no test that verifies an LRA is cancelled if a participant tries to join with an LRA and fails because an external coordinator is unavailable. -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3204) Upgrade LRA TCK recovery handling to accommodate specification changes

by Thomas Jenkinson (Jira)

[ https://issues.redhat.com/browse/JBTM-3204?page=com.atlassian.jira.plugin... ] Thomas Jenkinson commented on JBTM-3204: ---------------------------------------- I thought we would mark them as resolved but no fix version, we can find them under https://issues.redhat.com/issues/?jql=project%20%3D%20JBTM%20AND%20status... > Upgrade LRA TCK recovery handling to accommodate specification changes > ----------------------------------------------------------------------- > > Key: JBTM-3204 > URL: https://issues.redhat.com/browse/JBTM-3204 > Project: JBoss Transaction Manager > Issue Type: Bug > Components: LRA > Affects Versions: 5.9.8.Final > Reporter: Martin Stefanko > Assignee: Martin Stefanko > Priority: Major > Fix For: 5.next > > > LRA TCK added a new way of handling LRA recovery in https://github.com/eclipse/microprofile-lra/commit/52b799768b3d0fa1746f2b.... Update the implementation to accommodate these changes. > Also update the LRA versions to the latest snapshots: > <version.microprofile.lra.api>1.0-20190927.061209-550</version.microprofile.lra.api> > <version.microprofile.lra.tck>1.0-20190927.061211-550</version.microprofile.lra.tck> > diff: [https://github.com/eclipse/microprofile-lra/compare/1.0-RC1...fab960eede3...] -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3229) Allow to enlist AfterLRA listeners after the LRA enters the ending phase

by Thomas Jenkinson (Jira)

[ https://issues.redhat.com/browse/JBTM-3229?page=com.atlassian.jira.plugin... ] Thomas Jenkinson commented on JBTM-3229: ---------------------------------------- [~mmusgrov] Can this be resolved now? > Allow to enlist AfterLRA listeners after the LRA enters the ending phase > ------------------------------------------------------------------------ > > Key: JBTM-3229 > URL: https://issues.redhat.com/browse/JBTM-3229 > Project: JBoss Transaction Manager > Issue Type: Bug > Components: LRA > Affects Versions: 5.10.1.Final > Reporter: Martin Stefanko > Assignee: Martin Stefanko > Priority: Major > > https://github.com/eclipse/microprofile-lra/issues/241 > The specification was updated to allow AfterLRA listeners (LRA resources without Compensate method but with the AfterLRA method) to enlist for AfterLRA notification even after the LRA enters the ending (closing/canceling) phase. -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

[JBoss JIRA] (JBTM-3227) LRA branch TckTests#TestForget is failing on our CI infrastructure

by Thomas Jenkinson (Jira)

[ https://issues.redhat.com/browse/JBTM-3227?page=com.atlassian.jira.plugin... ] Thomas Jenkinson commented on JBTM-3227: ---------------------------------------- [~mmusgrov] can this be marked resolved yet (without fix version though) as it is on LRA branch: https://github.com/jbosstm/narayana/commits/LRA > LRA branch TckTests#TestForget is failing on our CI infrastructure > ------------------------------------------------------------------ > > Key: JBTM-3227 > URL: https://issues.redhat.com/browse/JBTM-3227 > Project: JBoss Transaction Manager > Issue Type: Bug > Components: LRA, Testing > Reporter: Michael Musgrove > Assignee: Michael Musgrove > Priority: Major > > The LRA branch tracks MP-LRA changes and we test it on our CI set up. > The branch depends on snapshot builds but the current build is failing because the test TckTests#TestForget is wrong and it has been fixed in a later snapshot. Therefore we need to upgrade the snapshot build dependency -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 10 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

jbossts-issues January 2020