David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/628831#628831 -------------------------------------------------------------- I was thinking that the root node could simply register a Synchronization (and perhaps also an interposed Synchronization) with its local TM resources.  This allows it to direct the remote clients through these phases.  The question would be how to inflow these requests into the subordinate nodes (and their subordinate nodes, etc.). -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/628831#628831] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629306#629306 -------------------------------------------------------------- Just a quick note here that we can use other classes than XATerminator; there is no need for this particular code (i.e. the EJB server transaction inflow code) to be loosely coupled from the transaction manager, so we should implement the most direct and most correct solution, which probably means just bypassing XATerminator and using the APIs consumed by it.  In fact we are already planning on doing so because EJB invocation isn't done with "Work" which is presently the way that tasks are executed via JBossXATerminator instances. But yeah, adding these methods (and a protocol message to convey them) is what I mean by "a way to inflow these requests".  The question is, do we support interposed sync as well, which (as far as I understand) would mean two phases of each completion method (one to fire off non-interposed and one to fire off interposed)? Yeah that should be fine, just let me know what the method hooks will be.  I'm also fine with using SubordinationManager directly, or providing an alternative extended version of com.arjuna.ats.internal.jta.transaction.arjunacore.jca.XATerminatorImple, or whatever is easiest for you guys.  In particular, at a protocol level it does us no good to have all the execptions mapped to XAException because we'll just have to decode it again and send it over the wire; it just costs an extra, useless transaction wrapping. Yeah the current mechanism involves getting an XID how you describe (via the client registering itself as an XAResource) and attaching the resultant XID to all requests in that transaction; the server-side interceptor will be responsible for mapping the XID to a Transaction to resume for the duration of the request.  I think we probably should use the same SubordinationManager-based scheme for looking up the transaction though. Yeah this is the #1 issue.  I think the only real option is a new XID format. Since the node relationship is hierarchical, it makes sense to me that the bqual should also be hierarchical.  We could use a single integer value to represent the branch relative to a particular node, and keep a local mapping of value to subnode identification.  This would be enough information to recover, as long as the node connections are all active. Using fixed-length integers, 64 bytes limits things considerably.  Even if you didn't need any bytes for recording existing information that gives you at most 64 levels of 256 resources per level. You could however use integer packing to make more room; in such a scheme, the bqual bits which identify the node would still be an integer sequence but would work like this: * Values 0000-0111 (0-7): resources 0-7, length is 4 bits * Values 1000 0000-1011 1111 (0x80-0xBF): resources 8-71, length is 8 bits * Values 1100 0000 0000-1101 1111 1111 (0xC00-0xDFF): resources 72-583, length is 12 bits * etc. You can tell how many bits to consume by the number of leading high-order bits.  Such a scheme can be expanded indefinitely, and supports both many levels with few resources and few levels with many resources. I assume that as part of recovery, we will have to somehow recreate all of our XAResources?  Or does the TM serialize them or something? -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629306#629306] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629394#629394 -------------------------------------------------------------- David We need to be really careful discussing interposed syncs, because the term is overloaded and hence ambiguous. In the context of JTA 1.1, TransactionSynchronizationRegistry.registerInterposedSynchronization has specific partial ordering guarantees. However, as the JTA does not explicitly cover distributed tx semantics, those ordering guarantees are relative to the other Synchronizations in the same JVM only. We may or may not wish to offer wider guarantees or config options according to how useful they are and how easy they are to implement. The term 'interposition' also has meaning in distributed tx: a subordinate placed between the root coordinator and eventual child resource or Synchronization to form a tree structure.  There are several ways of implementing that, including: Synchronization objects may only be registered with their local node and are driven as part of that local node's beforeCompletion phase. That phase may or may not itself be interposed to the corresponding phase in the root coordinator.  Alternatively, the local Synchronization may be wrapped in a remote transport aware proxy and registered with the root (or an intermediate parent if the tree is multi level) coordinator. In that case the beforeCompletion phase in the child node is a no-op, the local Synchronization list being empty. Naturally all three solutions display subtly different semantics and not so subtly different performance characteristics. The final piece of background you need before we continue on to implementation choices is this: the transaction engine underpinning the JTA is not directly aware of the distinction between JTA 'interposed' and regular Synchronizations. It does not maintain separate lists for them, nor process them in separate phases. The necessary ordering guarantees are achieved by sorting a single list of heterogeneous Synchronizations elements using, amongst other things, their 'interposed' attribute. This impacts how easy it is to expose separate lifecycle phases so the subordinate tx's lifecyle can be driven in a more fine grained manner. Specifically, it's easy to expose beforeCompletion and have it run all the Syncs, but considerably harder to expose separate method for running regular and JTA-interposed syncs. So, the options are: Wrap the local subordinate tx object supplied by TS with an object that intercepts the registerSynchronization calls and handles them itself, e.g. by proxying them individually or as a group to the parent coordinator. Work needed on the TS side: none i.e. you won't be held up waiting for us. IIRC various AS component already do something like this for a local JVM case where they need ordering guarantees between their own Syncs that exceed what can be done with JTA semantics. Register Syncs with the local subordinate and register a single Sync in the parent, using the impl thereof to pass the beforeCompletion call down to the subordinate coordinator. Work needed on the TS side: none for beforeCompletion as it's already exposed. Some to expose afterCompletion and allow some way for commit|rollback to indicate afterCompletion processing should be deferred until an explicit call for it is made, as they currently bundle it in automatically and must continue to do so in the default JCA case. Allow JTA 1.1. 'interposition' ordering guarantees to span multiple nodes: Do it using the first method above, except you either register two Syncs proxies (one regular, one 'interposed') in the parent, or you register every child Sync in the remote parent using its own individual proxy, using a type appropriate wrapper to preserve ordering. Work needed on the TS side: none.  Do it using method two above: lots, as the TS core would need to be refactored to support handing multiple sync phases separately before appropriate methods could be exposed for you to call. Don't hold your breath. The joker in the pack is of course the failure semantics. In JTA all Synchronizations are volatile (another overloaded term. 'in-memory only' rather than 'memory fence aware' in this context). They are not logged to disk and hence afterCompletion calls won't run in the event of a crash. Where everything is in one JVM that is relatively easy to reason about. Where a crash may affect only one of the JVMs in an interposed hierarchy, then you need to start worrying about detecting and cleaning up orphaned afterCompletions that may never get called.   A typical afterCompletion call is 'release connection back to pool', so leaking them is going to suck big time. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629394#629394] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629409#629409 -------------------------------------------------------------- This is what I was attempting to describe above. That said, the limitation with this approach is clear, in the case where there are no synchronizations registered on the subordinate TM, we will still have to call each subordinate TM with a beforeCompletion message as the root TM will not be aware of which subordinates do/don't have sync's registered, a perfomance penalty. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629409#629409] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629519#629519 -------------------------------------------------------------- Hi David, If you are happy do wait for JTA interposed Synchronization behavior, but do eventually want the same, may I ask you to raise a Jira for it? You can assign it to me if you like and mark it is a dependency of: https://issues.jboss.org/browse/JBTM-895 https://issues.jboss.org/browse/JBTM-895. If you want, you can mark it as a fix for 4.15.x Tom -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629519#629519] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629593#629593 -------------------------------------------------------------- Thanks :) -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629593#629593] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629560#629560 -------------------------------------------------------------- It's hard to do lazily when all the communication is top down - there is no mechanism for the child to call up to the parent to register. You'd have to piggyback the registration on the return leg of a top down call rather than initiating a new bottom up one for it.  That's roughly one of the reasons I'm preferring to build the remoting transport around the jta rather than the jts. For example, the replay completion recovery model in jts won't work with remoting either - it needs the top down xa recovery scan model instead. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629560#629560] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629627#629627 -------------------------------------------------------------- We'd need to probably come up with some clever way to modify recovery processing, I can't think of any other way to solve this. Are we sure that we need afterCompletion synchronization processing to be executed in terms of the root controller?  The only time I can imagine it mattering is if it is controlling some resource that is outside of the JVM, outside of the transaction, and yet visible to other participants.  If synchronizations are used for the purpose of discovering the overall outcome of the transaction, then executing on the local node achieves that purpose.  Same thing if the purpose is to clean up some per-VM resource.  And I think these two use cases are by far the predominant ones. I would argue that it is actually more effective to define the behavior in terms of the single VM.  Especially considering that no other solution seems to be forthcoming... For our purposes, it might be best to come out and specify that afterCompletion only runs in terms of the individual VM's view of the overall transaction.  Then later on we can revisit having this behavior be configurable, if we actually find a good solution for how to perform this recovery. I know it's probably not an ideal solution, but I think it will cover 95% of the use cases correctly, and I suspect that the last 5% ought to be using proper XAResources of their own. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629627#629627] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629888#629888 -------------------------------------------------------------- That is correct.  However I argue that because of the non-durability of afterCompletions, this is no worse than the existing situation - anyone who relies on afterCompletion for the correctness of their transaction is subject to data loss if the VM crashes after the transaction is committed but before afteCompletion runs.  Because of this, as far as I see it, the only correct usage of afterCompletion is cleaning up local resources that don't survive a VM crash.  I just doesn't seem like a big loss to me. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629888#629888] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Mark Little [http://community.jboss.org/people/marklittle] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/630242#630242 -------------------------------------------------------------- No inconsistency. They're best effort. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/630242#630242] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Mark Little [http://community.jboss.org/people/marklittle] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/630237#630237 -------------------------------------------------------------- Yes piggybacking works too: we do that with interposition in JTS. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/630237#630237] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629477#629477 -------------------------------------------------------------- This is gonna bug me until I fix it... For this to work you'd have to treat 0000 as a special padding sequence for the case where only one nybble of the last byte is used.  Thus giving: * 0000: padding * 0001-0111: resources 0-6 * 1000 0000-1011 1111: resources 7-70 * 1100 0000 0000-1101 1111 1111: resources 71-582 * etc. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629477#629477] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629583#629583 -------------------------------------------------------------- Well, that API is already public so I think that ship has sailed.  That said I'd hate to pile more work on you guys in the name of Yet Another Abstraction (gods know we have enough of those already). Here's the deal with public API support. On the client side, our jboss-ejb-client JAR is expected to be forwards compatible from now until pretty much the end of time.  That means, no using non-standard transaction APIs, period - the "outflow" of the transaction is done purely via the TransactionManager and TransactionSynchronizationRegistry implementations and via XAResource, and I don't see this changing anytime soon.  The client cannot do anything that cannot be done with these interfaces. The server side (which needs to consume this API) is a completely different matter.  The wire protocol is our point of compatibility; everything else is tightly coupled to the AS release.  The classes which inflow the transaction into the server can be as proprietary and/or expedient as we need them to be - and given the late hour, it's not the time for any grand new abstraction layers.  We already have tight coupling with JBossTS - allowing pluggable transaction managers in AS 7 isn't something I see happening (and if it does, that would be the time to revisit abstraction layers). So what I'm saying is, I'd hate to see you guys dragged into a lot of unnecessary work when 95% of the required API already exists. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629583#629583] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/629604#629604 -------------------------------------------------------------- Yes, but it's public in that module. Whether or not an API is public is determined by wether it's public, not by whether it's being used in a particular codebase. There's just no reason to decouple this code, unless you really want to do the extra work.  From our perspective, if we used SubordinationManager directly, and then you made some incompatible change, we'd simply change our usage of it accordingly for the version of AS that included the updated TS; it's really not necessary to create an abstraction for our benefit.  The best API is not the one with the most abstractions, but the fewest - less code for everyone to maintain, and simpler contracts, mean a win for everyone in most cases.  (I could go on about API design all day...) I just want to make it absolutely, perfectly clear that this is not a requirement as far as we're concerned, we consider it to be much more important to hit the EAP 6 target than to have an abstraction layer over code which is already public (in the concrete Java sense, not in the socio-political sense you are alluding to).  It is up to you to decide if it is worth it. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/629604#629604] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/630539#630539 -------------------------------------------------------------- Hi David, Yeah, you will be responsible for recreating your XAResources, when they recover I believe you are intended to call RecoveryManagerService.addXAResourceRecovery() giving a class that implements org.jboss.tm.XAResourceRecovery from the transaction SPI. Tom -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/630539#630539] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion "Re: Remote Txn Inflow: Synchronizations" To view the discussion, visit: http://community.jboss.org/message/632701#632701 -------------------------------------------------------------- That's a bit over simplistic. beforeCompletion is not necessarily called on rollback only tx. Further, in distributed tx, rollback only is actually a local property - branches of the tx at some nodes may be considered rollback only whilst others in the same tx are not. So you can actually get weird situations in which beforeCompletion runs at some nodes but not others. Fun, isn't it. :-) see also JBTM-496 -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/632701#632701] Start a new discussion in JBoss Transactions Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]