Tooling delete operation can left hanging remote txn branch when it was subordinate prepared participant -------------------------------------------------------------------------------------------------------- Key: JBTM-3006 URL: https://issues.jboss.org/browse/JBTM-3006 Project: JBoss Transaction Manager Issue Type: Bug Components: Tooling Affects Versions: 5.8.0.Final Reporter: Ondra Chaloupka Assignee: Ondra Chaloupka Priority: Minor Attachments: data-subordinate.zip, jboss-cli-commands.txt, standalone-full.xml When administrator decides to call 'delete' in WFLY (or as jmx mbean call) and the transaction is subordinate (meaning started by another server and the context was propagated to this one) then if they are in heuristic state the transaction is removed from the narayana object store and from the RA. If the transaction is in prepared state then participant is removed from narayana object store (that could be influenced by settings of {{ignoreMBeanHeuristics}} property, see https://github.com/jbosstm/narayana/blob/master/ArjunaCore/arjuna/classes...) but the transaction branch at the RA is left untouched - [XAResource.forget|https://docs.oracle.com/javaee/7/api/javax/transaction/...] is called but it fails at the participant remote branch (see [XA spec|pubs.opengroup.org/onlinepubs/009680699/toc.pdf) - the XA spec permits calling {{forget}} only on heuristically finished branches but this is still {{prepared}}. The final outcome is that there is removed record from Narayana object store and left untouched in-doubt (prepared) remote branch. Such record should be handled by recovery (ideally) the {{SubordinateJTAXAResourceOrphanFilter}} votes for rollback but then the {{SubordinationManagerXAResourceOrphanFilter}} forces to leave it as it is. Thus such record will be handing forever. {code} TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() DEBUG [com.arjuna.ats.jta] (Periodic Recovery) No record found for < formatId=131077, gtrid_length=29, bqual_length=37, tx_uid=0:ffff0a280466:-51bd8cd:5aa92b2b:1c, node_name=1, branch_uid=0:ffff0a280466:758014d3:5aa92b2f:26, subordinatenodename=2, eis_name=java:/JmsXA NodeId:2bda713c-2790-11e8-8b4f-0a0027000000 > DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XAResourceOrphanFilter com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateJTAXAResourceOrphanFilter voted ROLLBACK DEBUG [com.arjuna.ats.jta] (Periodic Recovery) subordinate node name of < formatId=131077, gtrid_length=29, bqual_length=37, tx_uid=0:ffff0a280466:-51bd8cd:5aa92b2b:1c, node_name=1, branch_uid=0:ffff0a280466:758014d3:5aa92b2f:26, subordinatenodename=2, eis_name=java:/JmsXA NodeId:2bda713c-2790-11e8-8b4f-0a0027000000 > is 2 TRACE [com.arjuna.ats.jta] (Periodic Recovery) TransactionImple.getStatus: javax.transaction.Status.STATUS_UNKNOWN DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XAResourceOrphanFilter com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinationManagerXAResourceOrphanFilter voted LEAVE_ALONE TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@3397522a 1521650139598 1521650140600 true {code}