[JBoss JIRA] (JBTM-1369) Benchmark performance difference between JacORB and JDK ORB
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1369?page=com.atlassian.jira.plugin.... ]
Michael Musgrove edited comment on JBTM-1369 at 2/19/14 10:10 AM:
------------------------------------------------------------------
We have four JTS performance unit tests - they all run 1000 empty transactions with each test using a different method to begin the txn. I got the following figures from our CI cluster but if you want to try them on your own machine do the following:
Build narayana with JDK orb support enabled:
{code}./build.sh -Didlj-enabled=true clean install -DskipTests{code}
To run just a specific test, such as Performance2 for example:
{code}./build.sh -f ArjunaJTS/jts/pom.xml -Didlj-enabled=true -Dtest=com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance2{code}
View the results in the surefire report directories:
{code}JacORB: ArjunaJTS/jts/target/surefire-reports/com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance1-output.txt
JDK ORB: ArjunaJTS/jts/target/idlj-surefire-reports/com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance1-output.txt{code}
The results generally show jacorb performance as being significantly better on our CI cluster but, interestingly, on my laptop JDK orb seemed to do better:
{code:title=com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance1}
1000 txns (OTSImpleManager.current), no participants (with warm up of 10 txns):
idlj
Average time for empty transaction = 1.284000039100647
Transactions per second = 778.8161756602676
jacorb
Average time for empty transaction = 1.3229999542236328
Transactions per second = 755.8579248680498
{code}
{code:title=com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance2}
1000 txns (TSImpleManager.factory().create()), no participants (with warm up of 10 txns):
idlj
Average time for empty transaction = 3.48799991607666
Transactions per second = 286.6972546045273
jacorb
Average time for empty transaction = 2.115999937057495
Transactions per second = 472.5898061181409
{code}
{code:title=Test: com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance3}
1000 txns (ArjunaTransactionImple), no participants (with warm up of 10 txns):
idlj
Average time for empty transaction = 0.05400000140070915
Transactions per second = 18518.5180381656
jacorb
Average time for empty transaction = 0.052000001072883606
Transactions per second = 19230.76883399276
{code}
{code: title=Test: com.hp.mwtests.ts.jts.local.synchronizations.Performance}
1000 txns (org.omg.CosTransactions.Current) each registering a single synchronization
idlj
TPS: 191.20458891013382
jacorb
TPS: 309.2145949288807
{code}
was (Author: mmusgrov):
We have four JTS performance unit tests - they all run 1000 txns, no participants and use different methods to begin a transaction. The results generally show jacorb performance as being significantly better:
com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance1
1000 txns (OTSImpleManager.current), no participants (with warm up of 10 txns):
idlj
Average time for empty transaction = 1.284000039100647
Transactions per second = 778.8161756602676
jacorb
Average time for empty transaction = 1.3229999542236328
Transactions per second = 755.8579248680498
com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance2
1000 txns (TSImpleManager.factory().create()), no participants (with warm up of 10 txns):
idlj
Average time for empty transaction = 3.48799991607666
Transactions per second = 286.6972546045273
jacorb
Average time for empty transaction = 2.115999937057495
Transactions per second = 472.5898061181409
com.hp.mwtests.ts.jts.orbspecific.local.performance.Performance3
1000 txns (ArjunaTransactionImple), no participants (with warm up of 10 txns):
idlj
Average time for empty transaction = 0.05400000140070915
Transactions per second = 18518.5180381656
jacorb
Average time for empty transaction = 0.052000001072883606
Transactions per second = 19230.76883399276
com.hp.mwtests.ts.jts.local.synchronizations.Performance
1000 txns (org.omg.CosTransactions.Current) each registering a single synchronization
idlj
TPS: 191.20458891013382
jacorb
TPS: 309.2145949288807
> Benchmark performance difference between JacORB and JDK ORB
> -----------------------------------------------------------
>
> Key: JBTM-1369
> URL: https://issues.jboss.org/browse/JBTM-1369
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: JTS, Performance Testing
> Affects Versions: 4.17.0
> Reporter: Michael Musgrove
> Assignee: Gytis Trikleris
> Fix For: 5.0.2.Final
>
>
> JBTM-934 added support for the JDK orb to JTS. IOR sizes can range in size from 512 bytes to entirely unbounded, with the ORB being able to add arbitrary data within specific portions. Since the IOR is packed into transaction logs and if the IOR sizes are significantly different we may find that transaction throughput has been improved or degraded. We should run some benchmarks to see what the impact is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira