[JBoss JIRA] (JBTM-1444) Review the logging in XTS when participants throw Exceptions in phase 2 methods
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-1444?page=com.atlassian.jira.plugin.... ]
Gytis Trikleris updated JBTM-1444:
----------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/jbosstm/narayana/pull/479
> Review the logging in XTS when participants throw Exceptions in phase 2 methods
> -------------------------------------------------------------------------------
>
> Key: JBTM-1444
> URL: https://issues.jboss.org/browse/JBTM-1444
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: XTS
> Reporter: Paul Robinson
> Assignee: Gytis Trikleris
> Labels: assign
> Fix For: 5.0.0.Final
>
> Original Estimate: 1 day
> Time Spent: 1 day
> Remaining Estimate: 0 minutes
>
> To experiment:
> 1. Take the JDF WS-AT and WS-BA quickstarts.
> 2. Update the server logging to TRACE for the 'arjuna' catagory.
> 3. Observe what happens when the declared exceptions (and RuntimeException) are thrown from each of the phase two methods on the participants (close, compensate, commit, rollback).
> 3.1 We are interested in what logging is produced (and at what log level) and what behaviour it has on the application (does it hang, is an exception returned to the client, or something else).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1444) Review the logging in XTS when participants throw Exceptions in phase 2 methods
by Gytis Trikleris (JIRA)
[ https://issues.jboss.org/browse/JBTM-1444?focusedWorklogId=12429932&page=... ]
Gytis Trikleris logged work on JBTM-1444:
-----------------------------------------
Author: Gytis Trikleris
Created on: 02/Oct/13 9:19 AM
Start Date: 02/Oct/13 9:19 AM
Worklog Time Spent: 1 day
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 1 day)
Time Spent: 1 day
Worklog Id: (was: 12429932)
> Review the logging in XTS when participants throw Exceptions in phase 2 methods
> -------------------------------------------------------------------------------
>
> Key: JBTM-1444
> URL: https://issues.jboss.org/browse/JBTM-1444
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: XTS
> Reporter: Paul Robinson
> Assignee: Gytis Trikleris
> Labels: assign
> Fix For: 5.0.0.Final
>
> Original Estimate: 1 day
> Time Spent: 1 day
> Remaining Estimate: 0 minutes
>
> To experiment:
> 1. Take the JDF WS-AT and WS-BA quickstarts.
> 2. Update the server logging to TRACE for the 'arjuna' catagory.
> 3. Observe what happens when the declared exceptions (and RuntimeException) are thrown from each of the phase two methods on the participants (close, compensate, commit, rollback).
> 3.1 We are interested in what logging is produced (and at what log level) and what behaviour it has on the application (does it hang, is an exception returned to the client, or something else).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1638) QA suite failure: connection refused in JTSRemote_DistributedHammerTest3
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1638?page=com.atlassian.jira.plugin.... ]
Michael Musgrove edited comment on JBTM-1638 at 10/2/13 6:21 AM:
-----------------------------------------------------------------
The test failed again (see JBTM-1948). Re-opened for further consideration.
was (Author: mmusgrov):
The test failed again (see JBTM-1638). Re-opened for further consideration.
> QA suite failure: connection refused in JTSRemote_DistributedHammerTest3
> ------------------------------------------------------------------------
>
> Key: JBTM-1638
> URL: https://issues.jboss.org/browse/JBTM-1638
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Testing
> Affects Versions: 4.17.4
> Reporter: Gytis Trikleris
> Assignee: Michael Musgrove
> Priority: Minor
> Fix For: 4.17.10
>
>
> http://172.17.131.2/view/Narayana+BlackTie/job/jbossts-EAP61/1760
> client_0_output.txt:
> {code}
> starting command: /usr/local/jdk1.6.0_37/bin/java -classpath dist/jbossts-full-4.17.4.Final-SNAPSHOT/etc/:dist/jbossts-full-4.17.4.Final-SNAPSHOT/lib/*:dist/jbossts-full-4.17.4.Final-SNAPSHOT/lib/ext/*:ext/fscontext.jar:ext/providerutil.jar:ext/jboss-profiler-jvmti.jar:ext/jboss-logging-spi.jar:tests/build/classes/:dbdrivers/selected_dbdriver/*:dbdrivers/DB2_v9.7/db2jcc.jar:dbdrivers/jConnect-6_0/classes/jconn3.jar:dbdrivers/mssql2005_sqljdbc_2.0/enu/sqljdbc4.jar:dbdrivers/mysql-connector-java-5.1.8-bin.jar:dbdrivers/oracle_10_2_0_4/ojdbc14.jar:dbdrivers/postgresql-8.3-605.jdbc4.jar:ext/netty.jar -Dqa.debug=true -Djava.naming.provider.url=file:///tmp -Djava.naming.factory.initial=com.sun.jndi.fscontext.RefFSContextFactory -Dperformanceprofilestore.dir=config/perf_profiles/ -Djdbcprofilestore.dir=config/jdbc_profiles -Dmemorytestprofilestore.dir=config/memory_profiles/ -Dots.server.bindname=value_1 -DCoordinatorEnvironmentBean.maintainHeuristics=NO -DRecoveryEnvironmentBean.recoveryBackoffPeriod=5 -DCoreEnvironmentBean.timeoutFactor=2 -DCoordinatorEnvironmentBean.defaultTimeout=240 -Demma.coverage.out.file=./testoutput/jtsremote/JTSRemote_DistributedHammerTest3/client_0-coverage.ec -DportOffsetId=4 -DObjectStoreBaseDir=/home/hudson/workspace/jbossts-EAP61/qa/testoutput/jtsremote/JTSRemote_DistributedHammerTest3/client_0 -DRecoveryEnvironmentBean.recoveryListener=true com.hp.mwtests.ts.jts.remote.hammer.DistributedHammer3 value_1 value_2
> 2013-04-16 17:55:29,152 err: log4j:WARN No appenders could be found for logger (org.jboss.logging).
> 2013-04-16 17:55:29,152 err: log4j:WARN Please initialize the log4j system properly.
> 2013-04-16 17:55:29,717 out: 0 begin get12
> 2013-04-16 17:55:29,803 out: 0 part1 get12 : true
> 2013-04-16 17:55:29,864 out: 0 part2 get12 : true
> 2013-04-16 17:55:29,928 out: 0 end 0 get12 : true : true : true : 10 : 101
> 2013-04-16 17:55:29,929 out: 0 begin get21
> 2013-04-16 17:55:29,954 out: 0 part1 get21 : true
> 2013-04-16 17:55:29,983 out: 0 part2 get21 : true
> 2013-04-16 17:55:30,034 out: 0 end 0 get21 : true : true : true : 101 : 10
> 2013-04-16 17:55:30,035 out: Tripling the timeout from: 240
> 2013-04-16 17:55:30,035 out: 1 fork
> 2013-04-16 17:55:30,035 out: Tripling the timeout from: 240
> 2013-04-16 17:55:30,036 out: 2 begin incr21
> 2013-04-16 17:55:30,051 out: 3 begin
> 2013-04-16 17:55:30,053 out: 3 begin
> 2013-04-16 17:55:30,070 out: 2 part1 incr21 : true
> 2013-04-16 17:55:30,088 out: 3 begin get21
> 2013-04-16 17:55:30,100 out: 3 begin get12
> 2013-04-16 17:55:30,135 out: 2 part2 incr21 : true
> 2013-04-16 17:55:30,163 out: 2 end 3 part1 get21 : false
> 2013-04-16 17:55:30,167 out: 3 abort 3 get21 : false : false : false : 0 : 0
> 2013-04-16 17:55:30,178 out: 3 3 part1 get12 : false
> 2013-04-16 17:55:30,182 out: 3 abort 3 get12 : false : false : false : 0 : 0
> 2013-04-16 17:55:30,184 out: begin incr21
> 2013-04-16 17:55:30,210 out: 3 begin incr12
> 2013-04-16 17:55:30,222 out: 3 part1 incr21 : false
> 2013-04-16 17:55:30,239 out: 3 abort 3 incr21 : false : false : false : 3
> 2013-04-16 17:55:30,244 out: 3 abort
> 2013-04-16 17:55:30,250 out: 3 part1 incr12 : false
> 2013-04-16 17:55:30,254 out: 3 abort 3 incr12 : false : false : false : -12
> 2013-04-16 17:55:30,258 out: 3 end
> 2013-04-16 17:55:30,258 out: 1 join
> 2013-04-16 17:55:30,258 out: Tripling the timeout from: 720
> 2013-04-16 17:55:30,259 out: Tripling the timeout from: 2160
> 2013-04-16 17:55:30,259 out: 1 begin get12
> 2013-04-16 17:55:30,280 out: 1 part1 get12 : false
> 2013-04-16 17:55:30,284 out: 1 abort 1 get12 : false : false : false : 0 : 0
> 2013-04-16 17:55:30,284 out: Tripling the timeout from: 6480
> 2013-04-16 17:55:30,284 out: Tripling the timeout from: 19440
> 2013-04-16 17:55:30,284 out: 1 begin get12
> 2013-04-16 17:55:30,303 out: 1 part1 get12 : false
> 2013-04-16 17:55:30,311 out: 1 abort 1 get12 : false : false : false : 0 : 0
> 2013-04-16 17:55:30,311 out: Tripling the timeout from: 58320
> 2013-04-16 17:55:30,311 out: 1 begin incr21
> 2013-04-16 17:55:30,337 out: 1 part1 incr21 : false
> 2013-04-16 17:55:30,342 out: 1 abort 1 incr21 : false : false : false : -4
> 2013-04-16 17:55:30,342 out: Tripling the timeout from: 174960
> 2013-04-16 17:55:30,342 out: 1 begin
> 2013-04-16 17:55:30,348 out: 1 begin get12
> 2013-04-16 17:55:30,374 out: 1 part1 get12 : false
> 2013-04-16 17:55:30,378 out: 1 abort 1 get12 : false : false : false : 0 : 0
> 2013-04-16 17:55:30,381 out: 1 end
> 2013-04-16 17:55:30,381 out: Tripling the timeout from: 524880
> 2013-04-16 17:55:30,381 out: Tripling the timeout from: 1574640
> 2013-04-16 17:55:30,382 out: 1 begin
> 2013-04-16 17:55:30,389 out: 1 begin get21
> 2013-04-16 17:55:30,416 out: 2 incr21 : true : true : true : 9
> 2013-04-16 17:55:30,417 out: Tripling the timeout from: 720
> 2013-04-16 17:55:30,417 out: Tripling the timeout from: 2160
> 2013-04-16 17:55:30,417 out: 2 fork
> 2013-04-16 17:55:30,419 out: 4 begin
> 2013-04-16 17:55:30,422 out: 5 begin
> 2013-04-16 17:55:30,426 out: 5 abort
> 2013-04-16 17:55:30,429 out: 1 part1 get21 : true
> 2013-04-16 17:55:30,442 out: 4 begin get21
> 2013-04-16 17:55:30,485 out: 1 part2 get21 : true
> 2013-04-16 17:59:30,420 out: 1 end 2013-04-16 17:59:30,420 [Transaction Reaper] WARN com.arjuna.ats.arjuna - ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac118314:baef:516d8281:46 in state RUN
> 2013-04-16 17:59:30,422 out: 2013-04-16 17:59:30,420 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012095: Abort of action id 0:ffffac118314:baef:516d8281:46 invoked while multiple threads active within it.
> 2013-04-16 17:59:30,422 out: 2013-04-16 17:59:30,421 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012108: CheckedAction::check - atomic action 0:ffffac118314:baef:516d8281:46 aborting with 1 threads active!
> 2013-04-16 17:59:30,422 out: 2013-04-16 17:59:30,421 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012098: Abort of action id 0:ffffac118314:baef:516d8281:46 invoked while child actions active
> 2013-04-16 17:59:30,422 out: 2013-04-16 17:59:30,421 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012099: Aborting child: 0:ffffac118314:baef:516d8281:47
> 2013-04-16 17:59:30,422 out: 2013-04-16 17:59:30,421 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012095: Abort of action id 0:ffffac118314:baef:516d8281:47 invoked while multiple threads active within it.
> 2013-04-16 17:59:30,422 out: 2013-04-16 17:59:30,421 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012108: CheckedAction::check - atomic action 0:ffffac118314:baef:516d8281:47 aborting with 1 threads active!
> 2013-04-16 17:59:30,920 out: 2013-04-16 17:59:30,920 [Transaction Reaper] WARN com.arjuna.ats.arjuna - ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac118314:baef:516d8281:46 in state CANCEL
> 2013-04-16 17:59:31,421 out: 2013-04-16 17:59:31,420 [Transaction Reaper] WARN com.arjuna.ats.arjuna - ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac118314:baef:516d8281:46 in state CANCEL_INTERRUPTED
> 2013-04-16 17:59:31,421 out: 2013-04-16 17:59:31,421 [Transaction Reaper] WARN com.arjuna.ats.arjuna - ARJUNA012120: TransactionReaper::check worker Thread[Transaction Reaper Worker 0,5,main] not responding to interrupt when cancelling TX 0:ffffac118314:baef:516d8281:46 -- worker marked as zombie and TX scheduled for mark-as-rollback
> 2013-04-16 17:59:31,421 out: 2013-04-16 17:59:31,421 [Transaction Reaper] WARN com.arjuna.ats.arjuna - ARJUNA012110: TransactionReaper::check successfuly marked TX 0:ffffac118314:baef:516d8281:46 as rollback only
> 2013-04-16 18:03:28,719 out: 2013-04-16 18:03:28,719 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012092: Nested abort of action 0:ffffac118314:baef:516d8281:47 received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jts.resources.ResourceRecord
> 2013-04-16 18:03:28,719 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 208 completed: Maybe
> 2013-04-16 18:03:28,720 out: 4 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,720 out: 2013-04-16 18:03:28,719 [Thread-3] WARN com.arjuna.ats.arjuna - ARJUNA012073: BasicAction.End() - prepare phase of action-id 0:ffffac118314:baef:516d8281:40 failed.
> 2013-04-16 18:03:28,720 out: 2013-04-16 18:03:28,720 [Thread-3] WARN com.arjuna.ats.arjuna - ARJUNA012075: Action Aborting
> 2013-04-16 18:03:28,722 err: org.omg.CORBA.INVALID_TRANSACTION: vmcid: 0x0 minor code: 0 completed: No
> 2013-04-16 18:03:28,722 out: 2 join
> 2013-04-16 18:03:28,723 out: Tripling the timeout from: 6480
> 2013-04-16 18:03:28,723 out: Tripling the timeout from: 19440
> 2013-04-16 18:03:28,723 out: 2 begin incr12
> 2013-04-16 18:03:28,729 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,729 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,729 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,729 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,729 err: at org.omg.CosTransactions._SubtransactionAwareResourceStub.rollback_subtransaction(_SubtransactionAwareResourceStub.java:37)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.internal.jts.resources.ResourceRecord.nestedAbort(ResourceRecord.java:274)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2855)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Abort(BasicAction.java:1917)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1494)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.commit(ArjunaTransactionImple.java:375)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.internal.jts.ControlWrapper.commit(ControlWrapper.java:243)
> 2013-04-16 18:03:28,729 err: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.commit(CurrentImple.java:247)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:274)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:356)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,729 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,729 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,729 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,729 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,729 err: ... 18 more
> 2013-04-16 18:03:28,729 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,729 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,729 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,729 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,729 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.incr(_HammerStub.java:18)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.incr12(DistributedHammerWorker3.java:62)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:302)
> 2013-04-16 18:03:28,729 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,729 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,730 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,730 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,730 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,730 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,730 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,730 err: ... 9 more
> 2013-04-16 18:03:28,730 err: DistributedHammerWorker3.incr12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,730 out: 2 incr12 : false : false : false : -13
> 2013-04-16 18:03:28,730 out: Tripling the timeout from: 58320
> 2013-04-16 18:03:28,730 out: Tripling the timeout from: 174960
> 2013-04-16 18:03:28,730 out: Tripling the timeout from: 524880
> 2013-04-16 18:03:28,730 out: Tripling the timeout from: 1574640
> 2013-04-16 18:03:28,730 out: 2 fork
> 2013-04-16 18:03:28,731 out: 2013-04-16 18:03:28,731 [Thread-3] WARN com.arjuna.ats.arjuna - ARJUNA012092: Nested abort of action 0:ffffac118314:baef:516d8281:40 received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jts.resources.ResourceRecord
> 2013-04-16 18:03:28,733 out: 4 begin
> 2013-04-16 18:03:28,734 out: 4 end
> 2013-04-16 18:03:28,735 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,735 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,736 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,736 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,736 err: at org.omg.CosTransactions._SubtransactionAwareResourceStub.rollback_subtransaction(_SubtransactionAwareResourceStub.java:37)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.internal.jts.resources.ResourceRecord.nestedAbort(ResourceRecord.java:274)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2855)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Abort(BasicAction.java:1917)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1494)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.commit(ArjunaTransactionImple.java:375)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.internal.jts.ControlWrapper.commit(ControlWrapper.java:243)
> 2013-04-16 18:03:28,736 err: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.commit(CurrentImple.java:247)
> 2013-04-16 18:03:28,736 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:274)
> 2013-04-16 18:03:28,736 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,736 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:356)
> 2013-04-16 18:03:28,736 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,736 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,736 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,736 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,736 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,736 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,736 err: ... 18 more
> 2013-04-16 18:03:28,736 err: DistributedHammerWorker3.get21: org.omg.CORBA.TRANSACTION_ROLLEDBACK: vmcid: 0xc000 minor code: 849 completed: No
> 2013-04-16 18:03:28,738 out: 2013-04-16 18:03:28,735 [Thread-3] WARN com.arjuna.ats.arjuna - ARJUNA012092: Nested abort of action 0:ffffac118314:baef:516d8281:40 received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jts.resources.ResourceRecord
> 2013-04-16 18:03:28,738 out: 1 get21 : false : false : false : 110 : 1
> 2013-04-16 18:03:28,738 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,738 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,738 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,738 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,738 err: at org.omg.CosTransactions._ResourceStub.rollback(_ResourceStub.java:42)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.internal.jts.resources.ResourceRecord.topLevelAbort(ResourceRecord.java:391)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1625)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.rollback(ArjunaTransactionImple.java:482)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.internal.jts.ControlWrapper.rollback(ControlWrapper.java:282)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.internal.jts.ControlWrapper.cancel(ControlWrapper.java:182)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> 2013-04-16 18:03:28,738 err: at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> 2013-04-16 18:03:28,738 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,738 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,738 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,738 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,738 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,738 err: ... 15 more
> 2013-04-16 18:03:28,739 out: 2013-04-16 18:03:28,738 [Transaction Reaper Worker 0] WARN com.arjuna.ats.jts - ARJUNA022230: ResourceRecord.topLevelAbort caught exception
> 2013-04-16 18:03:28,739 out: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,739 out: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,739 out: at org.omg.CosTransactions._ResourceStub.rollback(_ResourceStub.java:42)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.internal.jts.resources.ResourceRecord.topLevelAbort(ResourceRecord.java:391)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1625)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.rollback(ArjunaTransactionImple.java:482)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.internal.jts.ControlWrapper.rollback(ControlWrapper.java:282)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.internal.jts.ControlWrapper.cancel(ControlWrapper.java:182)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> 2013-04-16 18:03:28,739 out: at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> 2013-04-16 18:03:28,739 out: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,739 out: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,739 out: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,739 out: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,739 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,739 out: ... 15 more
> 2013-04-16 18:03:28,740 out: 2013-04-16 18:03:28,740 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012091: Top-level abort of action 0:ffffac118314:baef:516d8281:46 received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jts.resources.ResourceRecord
> 2013-04-16 18:03:28,741 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,741 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,741 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,741 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,741 err: at org.omg.CosTransactions._ResourceStub.rollback(_ResourceStub.java:42)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.internal.jts.resources.ResourceRecord.topLevelAbort(ResourceRecord.java:391)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1625)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.rollback(ArjunaTransactionImple.java:482)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.internal.jts.ControlWrapper.rollback(ControlWrapper.java:282)
> 2013-04-16 18:03:28,741 err: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.rollback(CurrentImple.java:321)
> 2013-04-16 18:03:28,741 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:359)
> 2013-04-16 18:03:28,741 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,741 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,741 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,741 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,741 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,741 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,741 err: ... 15 more
> 2013-04-16 18:03:28,742 out: 2013-04-16 18:03:28,742 [Transaction Reaper Worker 0] WARN com.arjuna.ats.arjuna - ARJUNA012113: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] missed interrupt when cancelling TX 0:ffffac118314:baef:516d8281:46 -- exiting as zombie (zombie count decremented to 0)
> 2013-04-16 18:03:28,742 out: 2013-04-16 18:03:28,742 [Thread-3] WARN com.arjuna.ats.jts - ARJUNA022230: ResourceRecord.topLevelAbort caught exception
> 2013-04-16 18:03:28,742 out: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,742 out: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,742 out: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,742 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,742 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,743 out: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,743 out: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,743 out: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,743 out: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,743 out: at org.omg.CosTransactions._ResourceStub.rollback(_ResourceStub.java:42)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.internal.jts.resources.ResourceRecord.topLevelAbort(ResourceRecord.java:391)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1625)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.rollback(ArjunaTransactionImple.java:482)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.internal.jts.ControlWrapper.rollback(ControlWrapper.java:282)
> 2013-04-16 18:03:28,743 out: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.rollback(CurrentImple.java:321)
> 2013-04-16 18:03:28,743 out: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:359)
> 2013-04-16 18:03:28,743 out: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,743 out: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,743 out: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,743 out: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,743 out: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,743 out: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,743 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,743 out: ... 15 more
> 2013-04-16 18:03:28,743 out: 2013-04-16 18:03:28,743 [Thread-3] WARN com.arjuna.ats.arjuna - ARJUNA012091: Top-level abort of action 0:ffffac118314:baef:516d8281:3f received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jts.resources.ResourceRecord
> 2013-04-16 18:03:28,743 out: 5 begin
> 2013-04-16 18:03:28,754 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,754 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,754 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,754 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,754 err: at org.omg.CosTransactions._ResourceStub.rollback(_ResourceStub.java:42)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.internal.jts.resources.ResourceRecord.topLevelAbort(ResourceRecord.java:391)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1625)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.rollback(ArjunaTransactionImple.java:482)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.internal.jts.ControlWrapper.rollback(ControlWrapper.java:282)
> 2013-04-16 18:03:28,754 err: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.rollback(CurrentImple.java:321)
> 2013-04-16 18:03:28,754 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:359)
> 2013-04-16 18:03:28,754 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,754 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,754 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,754 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,754 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,754 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,754 err: ... 15 more
> 2013-04-16 18:03:28,755 out: 5 2013-04-16 18:03:28,754 [Thread-3] WARN com.arjuna.ats.jts - ARJUNA022230: ResourceRecord.topLevelAbort caught exception
> 2013-04-16 18:03:28,755 out: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,755 out: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,755 out: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,755 out: at org.omg.CosTransactions._ResourceStub.rollback(_ResourceStub.java:42)
> 2013-04-16 18:03:28,755 out: at com.arjuna.ats.internal.jts.resources.ResourceRecord.topLevelAbort(ResourceRecord.java:391)
> 2013-04-16 18:03:28,755 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> 2013-04-16 18:03:28,755 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2832)
> 2013-04-16 18:03:28,755 out: at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1625)
> 2013-04-16 18:03:28,755 out: at com.arjuna.ats.internal.jts.orbspecific.coordinator.ArjunaTransactionImple.rollback(ArjunaTransactionImple.java:482)
> 2013-04-16 18:03:28,756 out: at com.arjuna.ats.internal.jts.ControlWrapper.rollback(ControlWrapper.java:282)
> 2013-04-16 18:03:28,756 out: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.rollback(CurrentImple.java:321)
> 2013-04-16 18:03:28,756 out: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:359)
> 2013-04-16 18:03:28,756 out: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,756 out: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,756 out: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,756 out: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,756 out: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,756 out: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,756 out: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,756 out: ... 15 more
> 2013-04-16 18:03:28,756 out: 2013-04-16 18:03:28,755 [Thread-3] WARN com.arjuna.ats.arjuna - ARJUNA012091: Top-level abort of action 0:ffffac118314:baef:516d8281:3f received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jts.resources.ResourceRecord
> 2013-04-16 18:03:28,756 out: 1 abort
> 2013-04-16 18:03:28,757 out: Tripling the timeout from: 4723920
> 2013-04-16 18:03:28,757 out: 1 begin
> 2013-04-16 18:03:28,757 out: 1 abort
> 2013-04-16 18:03:28,757 out: Tripling the timeout from: 14171760
> 2013-04-16 18:03:28,757 out: 1 begin incr12
> 2013-04-16 18:03:28,758 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,758 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,758 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,758 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,758 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.incr(_HammerStub.java:18)
> 2013-04-16 18:03:28,758 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.incr12(DistributedHammerWorker3.java:62)
> 2013-04-16 18:03:28,758 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:302)
> 2013-04-16 18:03:28,758 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:54)
> 2013-04-16 18:03:28,758 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,758 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,758 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,758 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,758 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,758 err: ... 9 more
> 2013-04-16 18:03:28,759 err: DistributedHammerWorker3.incr12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,759 out: 1 incr12 : false : false : false : -3
> 2013-04-16 18:03:28,759 out: begin get21
> 2013-04-16 18:03:28,760 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,760 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,760 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,760 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,760 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,760 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,760 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,760 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:63)
> 2013-04-16 18:03:28,760 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,760 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,760 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,760 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,760 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,760 err: ... 9 more
> 2013-04-16 18:03:28,760 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,760 out: 5 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,761 out: Tripling the timeout from: 42515280
> 2013-04-16 18:03:28,761 out: 1 fork
> 2013-04-16 18:03:28,762 out: 5 end
> 2013-04-16 18:03:28,762 out: 2 join
> 2013-04-16 18:03:28,762 out: Tripling the timeout from: 4723920
> 2013-04-16 18:03:28,762 out: 2 fork
> 2013-04-16 18:03:28,766 out: 6 begin
> 2013-04-16 18:03:28,766 out: 7 begin
> 2013-04-16 18:03:28,772 out: 6 begin
> 2013-04-16 18:03:28,773 out: 8 begin
> 2013-04-16 18:03:28,784 out: 7 begin get21
> 2013-04-16 18:03:28,784 out: 6 begin incr21
> 2013-04-16 18:03:28,784 out: 6 begin get12
> 2013-04-16 18:03:28,785 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,785 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,785 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,785 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,785 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,785 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get12(DistributedHammerWorker3.java:187)
> 2013-04-16 18:03:28,785 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:314)
> 2013-04-16 18:03:28,785 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,785 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,785 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,785 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,785 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,785 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,785 err: ... 9 more
> 2013-04-16 18:03:28,785 err: DistributedHammerWorker3.get12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,785 out: 6 get12 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,787 out: 6 end
> 2013-04-16 18:03:28,788 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,788 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,788 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,788 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,788 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,788 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,788 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,788 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:63)
> 2013-04-16 18:03:28,788 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,788 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,788 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,788 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,788 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,788 err: ... 9 more
> 2013-04-16 18:03:28,789 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,789 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,789 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,789 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,789 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.incr(_HammerStub.java:18)
> 2013-04-16 18:03:28,789 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.incr21(DistributedHammerWorker3.java:125)
> 2013-04-16 18:03:28,789 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:308)
> 2013-04-16 18:03:28,789 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,789 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,789 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,789 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,789 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,789 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,789 err: ... 9 more
> 2013-04-16 18:03:28,789 err: DistributedHammerWorker3.incr21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,790 out: 6 incr21 : false : false : false : -14
> 2013-04-16 18:03:28,790 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,790 out: 6 abort
> 2013-04-16 18:03:28,790 out: 1 join
> 2013-04-16 18:03:28,790 out: Tripling the timeout from: 127545840
> 2013-04-16 18:03:28,791 out: Tripling the timeout from: 382637520
> 2013-04-16 18:03:28,791 out: 1 fork
> 2013-04-16 18:03:28,791 out: 8 begin get21
> 2013-04-16 18:03:28,791 out: 7 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,792 out: 7 end
> 2013-04-16 18:03:28,792 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,792 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,792 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,792 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,792 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,792 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,792 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,792 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,792 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,792 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,792 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,792 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,792 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,792 err: ... 9 more
> 2013-04-16 18:03:28,793 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,793 out: 8 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,802 out: 9 begin
> 2013-04-16 18:03:28,803 out: 7 begin
> 2013-04-16 18:03:28,811 out: 9 begin get12
> 2013-04-16 18:03:28,811 out: 8 begin get21
> 2013-04-16 18:03:28,812 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,812 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,812 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,812 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,812 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,812 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,812 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,812 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:63)
> 2013-04-16 18:03:28,812 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,812 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,812 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,812 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,812 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,812 err: ... 9 more
> 2013-04-16 18:03:28,813 out: 7 begin
> 2013-04-16 18:03:28,813 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,814 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,814 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,814 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,814 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,814 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,814 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get12(DistributedHammerWorker3.java:187)
> 2013-04-16 18:03:28,814 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:314)
> 2013-04-16 18:03:28,814 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,814 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,814 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,814 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,814 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,814 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,814 err: ... 9 more
> 2013-04-16 18:03:28,814 err: DistributedHammerWorker3.get12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,814 out: 8 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,815 out: 8 end
> 2013-04-16 18:03:28,815 out: 9 get12 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,819 out: 2 join
> 2013-04-16 18:03:28,819 out: Tripling the timeout from: 14171760
> 2013-04-16 18:03:28,819 out: Tripling the timeout from: 42515280
> 2013-04-16 18:03:28,819 out: 2 fork
> 2013-04-16 18:03:28,826 out: 10 begin
> 2013-04-16 18:03:28,828 out: 8 begin
> 2013-04-16 18:03:28,834 out: 9 begin get21
> 2013-04-16 18:03:28,835 out: 7 begin get12
> 2013-04-16 18:03:28,835 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,835 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,835 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,835 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,835 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,836 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,836 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,836 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,836 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:63)
> 2013-04-16 18:03:28,836 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,836 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,836 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,836 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,836 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,836 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,836 err: ... 9 more
> 2013-04-16 18:03:28,836 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,836 out: 9 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,836 out: 9 end
> 2013-04-16 18:03:28,837 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,837 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,837 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,837 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,837 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,837 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get12(DistributedHammerWorker3.java:187)
> 2013-04-16 18:03:28,837 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:314)
> 2013-04-16 18:03:28,837 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:356)
> 2013-04-16 18:03:28,837 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,837 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,837 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,837 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,837 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,837 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,837 err: ... 10 more
> 2013-04-16 18:03:28,841 err: DistributedHammerWorker3.get12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,841 out: 7 get12 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,844 out: 10 begin
> 2013-04-16 18:03:28,852 out: 8 begin get21
> 2013-04-16 18:03:28,853 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,853 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,853 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,853 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,853 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,853 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,853 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:320)
> 2013-04-16 18:03:28,853 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,853 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,853 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,853 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,853 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,853 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,853 err: ... 9 more
> 2013-04-16 18:03:28,853 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,853 out: 8 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,854 out: 8 abort
> 2013-04-16 18:03:28,858 out: 7 begin incr21
> 2013-04-16 18:03:28,858 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,858 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,858 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,858 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,858 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,858 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.incr(_HammerStub.java:18)
> 2013-04-16 18:03:28,858 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.incr21(DistributedHammerWorker3.java:125)
> 2013-04-16 18:03:28,858 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:308)
> 2013-04-16 18:03:28,858 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:357)
> 2013-04-16 18:03:28,858 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,858 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,858 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,859 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,859 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,859 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,859 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,859 err: ... 10 more
> 2013-04-16 18:03:28,860 err: DistributedHammerWorker3.incr21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,860 out: 10 begin incr12
> 2013-04-16 18:03:28,861 out: 7 incr21 : false : false : false : -12
> 2013-04-16 18:03:28,861 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,862 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,862 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,862 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,862 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.incr(_HammerStub.java:18)
> 2013-04-16 18:03:28,862 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.incr12(DistributedHammerWorker3.java:62)
> 2013-04-16 18:03:28,862 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:302)
> 2013-04-16 18:03:28,862 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:357)
> 2013-04-16 18:03:28,862 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:62)
> 2013-04-16 18:03:28,862 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,862 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,862 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,862 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,862 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,862 err: ... 10 more
> 2013-04-16 18:03:28,862 out: 7 abort
> 2013-04-16 18:03:28,863 err: DistributedHammerWorker3.incr12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,863 out: 10 incr12 : false : false : false : -2
> 2013-04-16 18:03:28,865 out: 10 abort
> 2013-04-16 18:03:28,866 out: 10 end
> 2013-04-16 18:03:28,866 out: 2 join
> 2013-04-16 18:03:28,866 out: Tripling the timeout from: 127545840
> 2013-04-16 18:03:28,866 out: Tripling the timeout from: 382637520
> 2013-04-16 18:03:28,867 err: Exception in thread "Thread-4" org.omg.CORBA.BAD_PARAM: vmcid: 0xa000 minor code: 4041 completed: No
> 2013-04-16 18:03:28,867 err: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.set_timeout(CurrentImple.java:443)
> 2013-04-16 18:03:28,867 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:52)
> 2013-04-16 18:03:28,867 out: Tripling the timeout from: 1147912560
> 2013-04-16 18:03:28,872 out: 7 begin incr21
> 2013-04-16 18:03:28,872 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,872 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,872 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,872 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,872 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.incr(_HammerStub.java:18)
> 2013-04-16 18:03:28,872 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.incr21(DistributedHammerWorker3.java:125)
> 2013-04-16 18:03:28,872 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.randomOperation(DistributedHammerWorker3.java:308)
> 2013-04-16 18:03:28,872 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3a.run(DHThreadObject3a.java:63)
> 2013-04-16 18:03:28,872 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,872 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,872 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,872 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,872 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,872 err: ... 9 more
> 2013-04-16 18:03:28,873 err: DistributedHammerWorker3.incr21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,873 out: 7 incr21 : false : false : false : 4
> 2013-04-16 18:03:28,873 out: 7 abort
> 2013-04-16 18:03:28,874 out: 1 join
> 2013-04-16 18:03:28,874 err: Exception in thread "Thread-3" org.omg.CORBA.BAD_PARAM: vmcid: 0xa000 minor code: 4041 completed: No
> 2013-04-16 18:03:28,874 err: at com.arjuna.ats.internal.jts.orbspecific.CurrentImple.set_timeout(CurrentImple.java:443)
> 2013-04-16 18:03:28,874 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DHThreadObject3b.run(DHThreadObject3b.java:52)
> 2013-04-16 18:03:28,874 out: Tripling the timeout from: 1147912560
> 2013-04-16 18:03:28,874 out: 0 begin get12
> 2013-04-16 18:03:28,876 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,876 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,876 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,876 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,876 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,876 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,876 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get12(DistributedHammerWorker3.java:187)
> 2013-04-16 18:03:28,876 err: at com.hp.mwtests.ts.jts.remote.hammer.DistributedHammer3.main(DistributedHammer3.java:106)
> 2013-04-16 18:03:28,877 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,877 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,877 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,877 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,877 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,877 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,877 err: ... 8 more
> 2013-04-16 18:03:28,877 err: DistributedHammerWorker3.get12: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,877 out: 0 get12 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,879 out: 0 begin get21
> 2013-04-16 18:03:28,880 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,880 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,880 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,880 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,880 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,880 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get21(DistributedHammerWorker3.java:249)
> 2013-04-16 18:03:28,880 err: at com.hp.mwtests.ts.jts.remote.hammer.DistributedHammer3.main(DistributedHammer3.java:107)
> 2013-04-16 18:03:28,880 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,880 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,880 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,880 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,880 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,880 err: ... 8 more
> 2013-04-16 18:03:28,880 err: DistributedHammerWorker3.get21: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,880 out: 0 get21 : false : false : false : 0 : 0
> 2013-04-16 18:03:28,886 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,886 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 48872"
> 2013-04-16 18:03:28,886 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,886 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,886 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,886 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get1(DistributedHammerWorker3.java:466)
> 2013-04-16 18:03:28,886 err: at com.hp.mwtests.ts.jts.remote.hammer.DistributedHammer3.main(DistributedHammer3.java:112)
> 2013-04-16 18:03:28,886 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,886 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,886 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,886 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,886 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,886 err: ... 8 more
> 2013-04-16 18:03:28,886 err: DistributedHammerWorker1.get1: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,890 err: Apr 16, 2013 6:03:28 PM com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl <init>
> 2013-04-16 18:03:28,890 err: WARNING: "IOP00410201: (COMM_FAILURE) Connection failure: socketType: IIOP_CLEAR_TEXT; hostname: 172.17.131.20; port: 38267"
> 2013-04-16 18:03:28,891 err: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2200)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(ORBUtilSystemException.java:2221)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:205)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:218)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(SocketOrChannelContactInfoImpl.java:101)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(CorbaClientRequestDispatcherImpl.java:171)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(CorbaClientDelegateImpl.java:118)
> 2013-04-16 18:03:28,891 err: at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:431)
> 2013-04-16 18:03:28,891 err: at com.hp.mwtests.ts.jts.TestModule._HammerStub.get(_HammerStub.java:60)
> 2013-04-16 18:03:28,891 err: at com.hp.mwtests.ts.jts.orbspecific.resources.DistributedHammerWorker3.get2(DistributedHammerWorker3.java:496)
> 2013-04-16 18:03:28,891 err: at com.hp.mwtests.ts.jts.remote.hammer.DistributedHammer3.main(DistributedHammer3.java:112)
> 2013-04-16 18:03:28,891 err: Caused by: java.net.ConnectException: Connection refused
> 2013-04-16 18:03:28,891 err: at sun.nio.ch.Net.connect(Native Method)
> 2013-04-16 18:03:28,891 err: at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
> 2013-04-16 18:03:28,891 err: at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(DefaultSocketFactoryImpl.java:60)
> 2013-04-16 18:03:28,891 err: at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(SocketOrChannelConnectionImpl.java:188)
> 2013-04-16 18:03:28,891 err: ... 8 more
> 2013-04-16 18:03:28,891 err: DistributedHammerWorker1.get2: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No
> 2013-04-16 18:03:28,891 out: failing: assertionFailed
> 2013-04-16 18:03:28,891 out: Failed
> 2013-04-16 18:03:28,891 out: failing: assertionFailed
> 2013-04-16 18:03:28,891 out: Failed
> 2013-04-16 18:03:28,892 out: Passed
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1951) Deadlock in JBOSS Transaction Reaper
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1951?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson resolved JBTM-1951.
---------------------------------
Resolution: Cannot Reproduce Bug
Resolving as cannot reproduce. Jesper feel free to move this to an IronJacamar issue and reopen if you feel it could apply to the latest version of the software.
> Deadlock in JBOSS Transaction Reaper
> ------------------------------------
>
> Key: JBTM-1951
> URL: https://issues.jboss.org/browse/JBTM-1951
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Affects Versions: 4.16.2
> Reporter: Peter Klauser
> Assignee: Tom Jenkinson
>
> Found one Java-level deadlock:
> =============================
> "Timer-93":
> waiting to lock monitor 0x00007f8fac190cf8 (object 0x00000000d6d50830, a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction),
> which is held by "Transaction Reaper Worker 0"
> "Transaction Reaper Worker 0":
> waiting for ownable synchronizer 0x00000000d6861408, (a java.util.concurrent.locks.ReentrantLock$FairSync),
> which is held by "Timer-93"
> Java stack information for the threads listed above:
> ===================================================
> "Timer-93":
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1573)
> - waiting to lock <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:188)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1247)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:134)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:70)
> at sun.reflect.GeneratedMethodAccessor1756.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)
> at com.sun.proxy.$Proxy852.afterDelivery(Unknown Source)
> at sunrise.ra.aq.inbound.AqServerSession.run(AqServerSession.java:141)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> "Transaction Reaper Worker 0":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000d6861408> (a java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock$FairSync.lock(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
> at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.lock(BaseWrapperManagedConnection.java:361)
> at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:290)
> at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:99)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:312)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2874)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1616)
> - locked <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:217)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> Found 1 deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1951) Deadlock in JBOSS Transaction Reaper
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1951?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson commented on JBTM-1951:
-------------------------------------
Hi Peter,
Its difficult to say without more complete understanding of your issue. As Jesper says it is worth trying this on the latest code base from WildFly and checking if the issue still occurs.
I had a look at how a ManagedConnection::lock() could be acquired and not released by Timer-93 and couldn't spot a scenario on the latest IronJacamar 1.1.x tag (unless you are subclassing BaseWrapperManagedConnection somehow?)
I will close the issue as cannot reproduce for now, if you do find an issue using WildFly, please open up a discussion on the ironjacamar forums
Thanks,
Tom
> Deadlock in JBOSS Transaction Reaper
> ------------------------------------
>
> Key: JBTM-1951
> URL: https://issues.jboss.org/browse/JBTM-1951
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Affects Versions: 4.16.2
> Reporter: Peter Klauser
> Assignee: Tom Jenkinson
>
> Found one Java-level deadlock:
> =============================
> "Timer-93":
> waiting to lock monitor 0x00007f8fac190cf8 (object 0x00000000d6d50830, a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction),
> which is held by "Transaction Reaper Worker 0"
> "Transaction Reaper Worker 0":
> waiting for ownable synchronizer 0x00000000d6861408, (a java.util.concurrent.locks.ReentrantLock$FairSync),
> which is held by "Timer-93"
> Java stack information for the threads listed above:
> ===================================================
> "Timer-93":
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1573)
> - waiting to lock <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:188)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1247)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:134)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:70)
> at sun.reflect.GeneratedMethodAccessor1756.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)
> at com.sun.proxy.$Proxy852.afterDelivery(Unknown Source)
> at sunrise.ra.aq.inbound.AqServerSession.run(AqServerSession.java:141)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> "Transaction Reaper Worker 0":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000d6861408> (a java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock$FairSync.lock(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
> at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.lock(BaseWrapperManagedConnection.java:361)
> at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:290)
> at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:99)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:312)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2874)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1616)
> - locked <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:217)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> Found 1 deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1951) Deadlock in JBOSS Transaction Reaper
by Peter Klauser (JIRA)
[ https://issues.jboss.org/browse/JBTM-1951?page=com.atlassian.jira.plugin.... ]
Peter Klauser commented on JBTM-1951:
-------------------------------------
Our code doing inbound MDB handling is a TimerTask which is scheduled to repeatedly run..
public void run()
{
log.info("AQ JCA - inbound dequeue thread started.");
MessageEndpoint endpoint = null;
shutdownRequest.set(false);
try {
initializeConnection(); // -- this is an oracle AQ connection > nothing to do with JCA/JDBC
MessageEndpointFactory endpointFactory = activation.getMessageEndpointFactory();
endpoint = endpointFactory.createEndpoint(xaResource);
dequeueOption = new AQDequeueOption();
while (!shutdownRequest.get()) {
try {
endpoint.beforeDelivery(AqActivation.ONMESSAGE);
AqMessage aqMessage = AqUtils.dequeue(getQueue(), getConfig().isSupportMessageSplit(), dequeueOption, getConfig().getCorrelationId(), getConfig().getDequeueTimeout());
dispatchRequest(endpoint, aqMessage);
} catch (AQException e) {
if (AqUtils.isEndOfQueueExcepion(e)) {
// ignore timeout
log.info("EOQE - nothing on the queue " + getQueue().getName());
} else {
log.error("AqException during dequeue:" + e, e);
shutdownRequest.set(true);
}
} catch (Throwable e) {
log.error("Handling Exception", e);
shutdownRequest.set(true);
} finally {
try{
endpoint.afterDelivery();
} catch(Exception rex){
log.error("Exception in afterDelivery(): ", rex);
shutdownRequest.set(true);
}
}
}
closeConnection(); // -- close the oracle AQ connection
if ( endpoint != null ) {
endpoint.release();
}
} catch ( Throwable t ) {
log.error("AQRA encountered runtime exception.", t);
} finally {
closeConnection(); // -- make sure the AQ connection is closed when the thread leaves.
}
}
> Deadlock in JBOSS Transaction Reaper
> ------------------------------------
>
> Key: JBTM-1951
> URL: https://issues.jboss.org/browse/JBTM-1951
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Affects Versions: 4.16.2
> Reporter: Peter Klauser
> Assignee: Tom Jenkinson
>
> Found one Java-level deadlock:
> =============================
> "Timer-93":
> waiting to lock monitor 0x00007f8fac190cf8 (object 0x00000000d6d50830, a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction),
> which is held by "Transaction Reaper Worker 0"
> "Transaction Reaper Worker 0":
> waiting for ownable synchronizer 0x00000000d6861408, (a java.util.concurrent.locks.ReentrantLock$FairSync),
> which is held by "Timer-93"
> Java stack information for the threads listed above:
> ===================================================
> "Timer-93":
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1573)
> - waiting to lock <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:188)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1247)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:134)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:70)
> at sun.reflect.GeneratedMethodAccessor1756.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)
> at com.sun.proxy.$Proxy852.afterDelivery(Unknown Source)
> at sunrise.ra.aq.inbound.AqServerSession.run(AqServerSession.java:141)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> "Transaction Reaper Worker 0":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000d6861408> (a java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock$FairSync.lock(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
> at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.lock(BaseWrapperManagedConnection.java:361)
> at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:290)
> at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:99)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:312)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2874)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1616)
> - locked <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:217)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> Found 1 deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1951) Deadlock in JBOSS Transaction Reaper
by Jesper Pedersen (JIRA)
[ https://issues.jboss.org/browse/JBTM-1951?page=com.atlassian.jira.plugin.... ]
Jesper Pedersen commented on JBTM-1951:
---------------------------------------
Please, retest on WildFly 8 - either master or Beta1+ - and open a forum post if the problem still persist
> Deadlock in JBOSS Transaction Reaper
> ------------------------------------
>
> Key: JBTM-1951
> URL: https://issues.jboss.org/browse/JBTM-1951
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Affects Versions: 4.16.2
> Reporter: Peter Klauser
> Assignee: Tom Jenkinson
>
> Found one Java-level deadlock:
> =============================
> "Timer-93":
> waiting to lock monitor 0x00007f8fac190cf8 (object 0x00000000d6d50830, a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction),
> which is held by "Transaction Reaper Worker 0"
> "Transaction Reaper Worker 0":
> waiting for ownable synchronizer 0x00000000d6861408, (a java.util.concurrent.locks.ReentrantLock$FairSync),
> which is held by "Timer-93"
> Java stack information for the threads listed above:
> ===================================================
> "Timer-93":
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1573)
> - waiting to lock <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:188)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1247)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:134)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:70)
> at sun.reflect.GeneratedMethodAccessor1756.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)
> at com.sun.proxy.$Proxy852.afterDelivery(Unknown Source)
> at sunrise.ra.aq.inbound.AqServerSession.run(AqServerSession.java:141)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> "Transaction Reaper Worker 0":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000d6861408> (a java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock$FairSync.lock(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
> at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.lock(BaseWrapperManagedConnection.java:361)
> at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:290)
> at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:99)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:312)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2874)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1616)
> - locked <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:217)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> Found 1 deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1951) Deadlock in JBOSS Transaction Reaper
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1951?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson commented on JBTM-1951:
-------------------------------------
>From what I can see this could relate to JCA so /cc [~jesper.pedersen]
What I believe is happening is that something on "Timer-93" in JCA has taken the lock on the managed connection before it calls abort.
In the "Transaction Reaper Worker 0" case, the lock is taken on the transaction, then the JCA lock is obtained for the managed connection.
The question is, why (and when) does Timer-93 need to hold: 0x00000000d6861408
For my own part I can say that the locking in the transaction manager is to prevent two threads calling cancel at once.
> Deadlock in JBOSS Transaction Reaper
> ------------------------------------
>
> Key: JBTM-1951
> URL: https://issues.jboss.org/browse/JBTM-1951
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Affects Versions: 4.16.2
> Reporter: Peter Klauser
> Assignee: Tom Jenkinson
>
> Found one Java-level deadlock:
> =============================
> "Timer-93":
> waiting to lock monitor 0x00007f8fac190cf8 (object 0x00000000d6d50830, a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction),
> which is held by "Transaction Reaper Worker 0"
> "Transaction Reaper Worker 0":
> waiting for ownable synchronizer 0x00000000d6861408, (a java.util.concurrent.locks.ReentrantLock$FairSync),
> which is held by "Timer-93"
> Java stack information for the threads listed above:
> ===================================================
> "Timer-93":
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1573)
> - waiting to lock <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:188)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1247)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:134)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:70)
> at sun.reflect.GeneratedMethodAccessor1756.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)
> at com.sun.proxy.$Proxy852.afterDelivery(Unknown Source)
> at sunrise.ra.aq.inbound.AqServerSession.run(AqServerSession.java:141)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> "Transaction Reaper Worker 0":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000d6861408> (a java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock$FairSync.lock(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
> at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.lock(BaseWrapperManagedConnection.java:361)
> at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:290)
> at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:99)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:312)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2874)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1616)
> - locked <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:217)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78)
> Found 1 deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1955) Review ALL documentation
by Mark Little (JIRA)
[ https://issues.jboss.org/browse/JBTM-1955?page=com.atlassian.jira.plugin.... ]
Mark Little commented on JBTM-1955:
-----------------------------------
I really hope we do support heuristics in XTS or we don't comply with the standard. That's what InconsistentInternalState is for. Yes it's not ideal, but it's better than what we had in the specification initially.
> Review ALL documentation
> ------------------------
>
> Key: JBTM-1955
> URL: https://issues.jboss.org/browse/JBTM-1955
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Documentation
> Reporter: Paul Robinson
> Assignee: Tom Jenkinson
> Fix For: 5.0.0.Final
>
>
> I think we need to read through all the documentation looking for:
> 1. Out of date stuff. For example, references to file locations in AS versions prior to AS7.
> 2. Irrelevant stuff. For example: XTS documents the theory of heuristics, despite having no support for them in the current protocols or implementation.
> 3. Incorrect stuff. No examples, but there could be some errors.
> 4. Missing stuff. Most likely for new features, not yet documented.
> For 1. in particular, we should search across the entire documentation for other occurrences of the same issue, once found. For example, if a reference to bin/run.sh is found, then grep over all the other docs for other occurrences.
> I think we should divide up the docs between multiple people so as to not over-burden a single individual with what is likely to be a tedious task.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0
[JBoss JIRA] (JBTM-1925) Enable remote debugger in JBossAS/WildFly for all tests
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-1925?page=com.atlassian.jira.plugin.... ]
Michael Musgrove updated JBTM-1925:
-----------------------------------
Fix Version/s: 5.0.0.Final
(was: 5.0.0.M5)
> Enable remote debugger in JBossAS/WildFly for all tests
> -------------------------------------------------------
>
> Key: JBTM-1925
> URL: https://issues.jboss.org/browse/JBTM-1925
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Paul Robinson
> Assignee: Gytis Trikleris
> Fix For: 4.17.11, 5.0.0.Final
>
>
> We need to enable the debugger in all arquillian.xml files. To do this:
> 1. Add this to each arquillian.xml:
> {code}
> <configuration>
> <property name="javaVmArguments">${server.jvm.args}</property>
> ....
> </configuration>
> {code}
> 2. Update the server.jvm.args in each maven profile for running the Arquillian tests:
> {code}
> <server.jvm.args>..... ${jvm.args.debug}</server.jvm.args>
> {code}
> Look at the XTS/localjunit/pom.xml for an example. You will notice that not all Arquillian profiles configure the server.jvm.args. In which case you will need to add the config to match the others.
> 3. Add this property to the root pom,xml:
> {code}
> <jvm.args.debug>-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n</jvm.args.debug>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 3 months
- 1
- 0