[JBoss JIRA] Created: (JBTM-840) AsynchStore.addWork() never leaves overlfow lock loop
by Tom Waterhouse (JIRA)
AsynchStore.addWork() never leaves overlfow lock loop
-----------------------------------------------------
Key: JBTM-840
URL: https://issues.jboss.org/browse/JBTM-840
Project: JBoss Transaction Manager
Issue Type: Bug
Security Level: Public (Everyone can see)
Affects Versions: 4.14.0
Environment: Ubuntu 10/Java 1.6/MySQL 5.5/JBoss JTS 4.14.0/Spring 3.0/Hibernate 3.6
Reporter: Tom Waterhouse
It looks like somehow when using CacheStore with JBoss JTA it is possible to get into an endless loop. On the server our app is deployed one cpu is pegged at 100% usage. A thread dump revealed that a thread is executing AsyncStore.addWork() and never leaves (thread dump shows line 330, _overflowLock.wait()).
Our cache size is set to 200k.
Here is from the thread dump:
"aggregation-1-14" prio=10 tid=0x00007f0508d0e800 nid=0x46e0 in Object.wait() [0x00007f050c8eb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.arjuna.ats.internal.arjuna.objectstore.AsyncStore.addWork(CacheStore.java:330)
- locked <0x00000007822a75f8> (a java.lang.Object)
at com.arjuna.ats.internal.arjuna.objectstore.CacheStore.write_state(CacheStore.java:116)
at com.arjuna.ats.internal.arjuna.objectstore.FileSystemStore.write_committed(FileSystemStore.java:134)
at com.arjuna.ats.arjuna.coordinator.BasicAction.prepare(BasicAction.java:2218)
- locked <0x0000000789f8f530> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1458)
- locked <0x0000000789f8f530> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:99)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:159)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1158)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:119)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1009)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy187.execute(Unknown Source)
at com.attensa.core.executor.AggregationCallableImpl.call(AggregationCallableImpl.java:106)
at com.attensa.core.executor.AggregationCallableImpl.call(AggregationCallableImpl.java:25)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] Created: (JBTM-527) Adding PMD to build system
by Romain PELISSE (JIRA)
Adding PMD to build system
---------------------------
Key: JBTM-527
URL: https://jira.jboss.org/jira/browse/JBTM-527
Project: JBoss Transaction Manager
Issue Type: Feature Request
Security Level: Public (Everyone can see)
Components: Build System
Affects Versions: future
Reporter: Romain PELISSE
Priority: Trivial
Attachments: adding-pmd-as-qa-tools.patch
I have a small patch that offers an integration of PMD (http://pmd.sourceforge.net/) into the current build system of JBoss Transaction. Obviously, this is not an highly important feature request, having PMD checking java code is just nice to have. Anyway, as your project is not likely to integrate this patch quickly, I also published the result of a PMD run on the current source code:
http://belaran.eu/jbosstm-pmd.html
(this way you can already check what PMD find out without setting up the tool itself)
I integrated PMD into the qa/ directory which seems the most appropriate directory at first glance... An other strategy would have been to define a "pmd" task inside some common build file ( maybe common/build.xml but it does not look like it), and then have each module call for this anttask with the appropriate java source folder... If you fell this approach is better, please let me know, i'll adapt my patch.
All "violation" detected by PMD are not relevant, a pmd report is just a set of hints of something that may be a bad idea or could be improved.
PMD rulesets (what kind of violation is looking for) are defined in the qa/pmd.xml. In this file you can easily include/exclude rules, and you can use it to configure the PMD Eclipse Plugin.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] Created: (JBTM-837) Transaction deadlock when NoSuchElementException is thrown from AsyncStore.doWork()
by Tom Waterhouse (JIRA)
Transaction deadlock when NoSuchElementException is thrown from AsyncStore.doWork()
-----------------------------------------------------------------------------------
Key: JBTM-837
URL: https://issues.jboss.org/browse/JBTM-837
Project: JBoss Transaction Manager
Issue Type: Bug
Security Level: Public (Everyone can see)
Affects Versions: 4.14.0
Environment: Ubuntu 64-bit Linux/JBoss JTA 4.14.0/Hibernate 3.6.1/Infinispan 3.4.0.CR2/Spring 3.0.5/MySQL 5.1
Reporter: Tom Waterhouse
Attachments: arjuna-deadlock-trace.txt, default-jbossts-properties.xml
Our product has a scheduled job that runs every 30 minutes. Callable instances are added to an ExecutorService instance which is currently configured for 16 threads.
Our JBoss JTA implementation is using CacheStore as the object store (default-jbossts-properties.xml attached).
After some time a NoSuchElementException is thrown from AsyncStore.doWork(). That exception is somehow blocking other transaction thread's execution.
The Java thread dump is attached as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] Created: (JBTM-817) Automate execution of XTS crash recovery tests
by Andrew Dinn (JIRA)
Automate execution of XTS crash recovery tests
----------------------------------------------
Key: JBTM-817
URL: https://jira.jboss.org/browse/JBTM-817
Project: JBoss Transaction Manager
Issue Type: Task
Security Level: Public (Everyone can see)
Components: XTS
Affects Versions: 4.13.0
Reporter: Andrew Dinn
Fix For: 4.15.0
The XTS codebase includes crash recovery tests in the sar/tests subtree which currently need to be executed manually. This needs to be changed so that the current tests can be run automatically via Hudson. After this has been completed it should then be possible to extend the test suite to exercise scenarios not yet covered by the current range of tests.
The tests themselves employ a web service client and several web services which are deployed to the app server with the test code. The behaviour of the client and web services can be scripted so this single deployment is capable of exercising all the scenarios which are required in order to lead up to a crash. However, automation is not straightforward for several reasons:
Execution of the tests requires starting up an app server twice, the first time so that it can be crashed at a specific point during execution then a second time so that recovery processing can be checked. The JBossTS codebase includes some utilities which can be used to help manage startup and shutdown of the JBoss AS instance.
Crashing of the app server and tracing of execution during the first and second runs requires the use of the Byteman agent and Byteman rules. Suitable Byteman rule scripts exist for all the current tests, However, trace output is currently written to a file and the output is verified by eyeball. Timing variations mean that this output does not always have a fixed format. Also, validation requires checking that identifiers printed during the first and second run match up. It would be worth investigating an alternative way of collecting and validating this trace information e.g. using the dtest package contributed to Byteman by Jonathan Halliday. dtest has been used to test similar scenarios in the JBossTS JTA - XTS bridge code (the latter is in the txbridge source tree).
Timing variations also mean that execution of recovery code may needs to be manipulated using Byteman rules in order to ensure that the circumstances specified in the test scenario are actually met. This can involve introducing delays or dropping messages to ensure that events are handled in a specific order. The existing Bbyteman scripts include rules to achieve this where needed by the current tests. However, once again, this complicates automation of the trace validation process since it requires some of the traced operations to be discounted until an occurrence which has been engineered is identified.
In the longer run the tests will need to be extended in two dimensions.
Firstly, the current test locates the client, web services and transaction coordinator in one application server. The client, web services and transaction coordinator need to be tested when they are deployed in different app servers in various possible combinations and the correct handling of a crash by one or more of these app servers needs to be validated.
Secondly, the current tests only test normal recovery situations. It will also be necessary to simulate failures in the recovery process, either by scripting the web service behaviour or by injecting faults using Byteman rules.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month
[JBoss JIRA] Created: (JBTM-673) Investigate other object store implementations
by Mark Little (JIRA)
Investigate other object store implementations
----------------------------------------------
Key: JBTM-673
URL: https://jira.jboss.org/jira/browse/JBTM-673
Project: JBoss Transaction Manager
Issue Type: Task
Security Level: Public (Everyone can see)
Components: Transaction Core
Reporter: Mark Little
Assignee: Mark Little
Fix For: 4.10.0
"... spin out messaging journal into separate component so we can consume it. Also look at other disk I/O alternatives, basically trying to batch syncs so one is shared by many tx rather than one per tx. Also maybe preliminary investigation of in-memory replication of logs via jgroups."
See related links. Need to re-check performance of transaction log and maybe make it the default implementation.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 1 month