MBean representing a transaction should consider state when is to be removed ---------------------------------------------------------------------------- Key: JBTM-3001 URL: https://issues.jboss.org/browse/JBTM-3001 Project: JBoss Transaction Manager Issue Type: Bug Components: Tooling Affects Versions: 5.8.0.Final Reporter: Ondra Chaloupka Assignee: Ondra Chaloupka Attachments: data-subordinate.zip, data-tooling.zip, standalone-full.xml When transaction mbean is to be removed by tooling - there is a transaction available in the object store and recovery does not catch it for deletion (the reason could be various - either the transaction is set with heuristic outcome, or transaction was subordinate and recovery waits for the outer player to finish transaction but there is nobody coming, or there is record in txn log store but the RA already forgot about transaction and we want to clean the transaction manager log store) - then currently the transaction manager just tries to call [XAResource.forget|https://docs.oracle.com/javaee/7/api/javax/transaction/...]. But the forget call is available (correct by the XA specification) only when the participant finished with in the heuristic state. But deleting transaction by the tooling (even that should be only the tool of the last resort as normally recovery manager handles this automatically) should cause the transaction and all participants are correctly finished (forget or rolled-back) even the transaction is in prepared state (not in heuristic one). When {{forget}} is called on prepared transaction branch then (e.g. for Artemis ActiveMQ) the {{XAException.XAER_NOTA}} is thrown[1]. * The mbean should consider the state of the participant and choose if to rollback ({{prepared}} state) or if to forget ({{heuristic}} state). * The mbean should consider to check {{XAException}} error code as when {{XAResource.forget}} returns {{XAER_NOTA}} (pubs.opengroup.org/onlinepubs/009680699/toc.pdf, {{The specified XID is not known by the resource manager as a heuristically completed XID.}}) then the branch is in incorrect state - probably it's prepared thus rollback could be run. {code} 2018-03-20 11:06:06,614 TRACE [com.arjuna.ats.jta] (management-handler-thread - 1) XAResourceRecord.forget for XAResourceRecord < resource:org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@25091a7a, txid:< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a2805b6:-19805d73:5ab0d062:1c, node_name=1, branch_uid=0:ffff0a2805b6:-19805d73:5ab0d062:23, subordinatenodename=null, eis_name=forgot eis name for: 1 >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ Artemis/2.0, jndiName: java:/JmsXA NodeId:c07b7f00-2c1e-11e8-9014-54e1ad661a0b com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@69af92f1 > 2018-03-20 11:06:06,636 WARN [com.arjuna.ats.jta] (management-handler-thread - 1) ARJUNA016006: XAResourceRecord forget failed: - forget threw exception: javax.transaction.xa.XAException at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.xaForget(ActiveMQSessionContext.java:490) at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.forget(ClientSessionImpl.java:1196) at org.apache.activemq.artemis.service.extensions.xa.recovery.ActiveMQXAResourceWrapper.forget(ActiveMQXAResourceWrapper.java:123) at org.apache.activemq.artemis.service.extensions.xa.ActiveMQXAResourceWrapperImpl.forget(ActiveMQXAResourceWrapperImpl.java:86) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.forget(XAResourceRecord.java:806) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.forgetHeuristic(XAResourceRecord.java:792) at com.arjuna.ats.arjuna.tools.osb.mbean.LogRecordWrapper.removeFromList(LogRecordWrapper.java:238) at com.arjuna.ats.arjuna.tools.osb.mbean.ActionBean$GenericAtomicActionWrapper.remove(ActionBean.java:463) at com.arjuna.ats.arjuna.tools.osb.mbean.ActionBean.remove(ActionBean.java:327) at com.arjuna.ats.arjuna.tools.osb.mbean.LogRecordWrapper.remove(LogRecordWrapper.java:218) at com.arjuna.ats.arjuna.tools.osb.mbean.ActionBean.remove(ActionBean.java:149) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71) at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at org.jboss.as.jmx.PluggableMBeanServerImpl$TcclMBeanServer.invoke(PluggableMBeanServerImpl.java:1475) at org.jboss.as.jmx.PluggableMBeanServerImpl.invoke(PluggableMBeanServerImpl.java:724) at org.jboss.as.txn.subsystem.LogStoreTransactionDeleteHandler.execute(LogStoreTransactionDeleteHandler.java:59) at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:982) at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:726) at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:450) at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1408) at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:423) at org.jboss.as.controller.ModelControllerImpl.lambda$execute$1(ModelControllerImpl.java:243) at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:263) at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:229) at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:243) at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler.doExecute(ModelControllerClientOperationHandler.java:240) at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler.access$400(ModelControllerClientOperationHandler.java:138) at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run(ModelControllerClientOperationHandler.java:162) at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run(ModelControllerClientOperationHandler.java:158) at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:287) at org.wildfly.security.auth.server.SecurityIdentity.runAs(SecurityIdentity.java:244) at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:254) at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:225) at org.jboss.as.controller.remote.ModelControllerClientOperationHandler$ExecuteRequestHandler$1.execute(ModelControllerClientOperationHandler.java:158) at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$1.doExecute(ManagementRequestContextImpl.java:70) at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$AsyncTaskRunner.run(ManagementRequestContextImpl.java:160) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) at java.lang.Thread.run(Thread.java:748) at org.jboss.threads.JBossThread.run(JBossThread.java:485) {code}