[jboss-dev-forums] [JBoss Transactions Development] - Re: Remote txinflow: XID changes

[Tom Jenkinson]

Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion

"Re: Remote txinflow: XID changes"

To view the discussion, visit: http://community.jboss.org/message/633431#633431

--------------------------------------------------------------
Hi David,

Sorry the example doesn't map directly to your use case, I condensed it from the test suite and it is really more guidance on what tasks are required from the transport so hopefully it has proved at least useful in that regard. Indeed as a talking point for questions at least it seems to have helped  :) 
NOTE: My answers are below but they are caveated by the statement that they are broadly based on the current approach.
> David Lloyd wrote:
> 
> > Tom Jenkinson wrote:
> > 
> > Just to round off the discussion:
> > 
> > Basically, I determined that the subordinate node name is not strictly a "node identifier" in the same sense as we are requiring it for a JTA gtrid value (which must be unique per object store). Instead it need only be unique per propagated transaction and the server does not need it to be the same value for different transactions flowed through the it, nor indeed must the value be globally unique (i.e. different servers can reuse the same id so long as it is not used by a different server within the scope of the same transaction).
> > 
> > All that that does need to happen is that the transport dynamically assign a unique integer for each node participating in a transaction, e.g. a counter starting at 1 and a recoverable mapping file be stored *by the transport* so that for any given transaction a server (keyed by the traditional String node identifier) knows what its own (now dynamic) "subordinate node name" was for the transaction, thereby assisting it with recovery.
> > 
> > To that end I have only had to change:
> > 
> > XID {
> > int:formatId
> > byte[]:gtrid { UID:sequence, String:nodeName }
> > byte[]:bqual {UID:sequence, String:EIS name }
> > }
> > 
> > To:
> > XID {
> > int:formatId
> > byte[]:gtrid { UID:sequence, String:nodeName }
> > byte[]:bqual {UID:sequence, int:subordinateNodeName, String:EIS name }
> > }
> > 
> > (i.e. added that single integer into the bqual).
> OK here's my first wave of dumb questions:
> 1. Why do you need a UID in the bqual?
> 2. EIS name in this case is the local node's unique name for the subordinate it's planning on talking to, right?
> 3. If so why do we also need subordinateNodeName?  Just to create a way to differentiate between the "first" inflow versus subsequent?
> 
Here is the answer to these questions:
Interesting suggestion! It is indeed possible that for this type of XA resource we could look at creating a bespoke XID with the format:
XID {
int:formatId
byte[]:gtrid { UID:sequence, String:nodeName }
byte[]:bqual {String:subordinateNodeName, String:parentNodeName }
}
We know we can do that because we know that there is only going to be (by convention) a single proxy to the remote server for this node, therefore the branch is unique without the Uid.

TS is not really geared up to support generating XIDs differently per XA resource, but we can probably change that!

Not to say we shouldn't do that, but I was thinking about this (well a close derivative of it, based on ints again sorry - but in my mind they are both tokens) over night though, there is still an issue with storing the node identifiers in the XID which the transport has to work around by persisting them.

Basically, if you did what you are saying (which is a neat idea). The proxy XA resource would be enlisted with the ID (assuming nodes named 1,2,3 etc - Strings though):
ProxyXAResource - bqual{"2","1"}
Remote subordinate transaction  - bqual{"3", "2"}

That means in recovery you can't ask the remote server for a list of XIDs it knows about because it will return {"3","2"} but the local server doesn't know about 3,2, in this scenario it knows about 2,1.

Also, "normal" XIDs would need to be able to fit a String subordinate node name in them (for orphan detection). Meaning that we would definitely need to make space for a normal bqual:
byte[]:bqual {UID:sequence, String:EISname}
As Jonathan suggested we can basically do this by changing EISname to be an int leading to:
byte[]:bqual {UID:sequence, String:subordinatenodename, int:EISnameKey}


> David Lloyd wrote:
> 
> > Tom Jenkinson wrote:
> > 
> > I do have examples and tests for all of this:   https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Final/atsintegration/tests https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Final/atsintegration/tests and  https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Final/atsintegration/examples https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Final/atsintegration/examples.
> > 
> > In particular, the example should prove to be extremely useful when developing the transport required for this as it shows and describes the points where relevant transaction data must be reliably persisted.
> > 
> > As I say, by reverting to a String node identifier, the solution is even closer to the original mechanism so therefore this new approach should have no impact on existing JTA/JCA work. The only difference now is that the space available to EIS namesis one byte shorter (from 36 to 35 bytes).
> OK I'm trying to make sense of this code now w.r.t. our stuff.  More dumb questions:
> 1. It really looks like you're "handing off" a transaction between nodes so that only one has the transaction at a time, but in the real world many nodes in the graph may be performing useful work at the same time (potentially even from multiple threads) under the same (global) transaction.  How does this impact this scheme (from a TM perspective, not a transport perspective)?
> 2. You talk about not committing at the root node... should we be tracking the transactions by gtid and detecting the circular flow at the transport layer?  I assume that if that is the case, we'd use the original XID instead of the new alias and we'd treat commit requests of the alias XID as a "no-op" sort of thing.
> 3. It seems to me that we'd only have to persist the XID and enough information to revive a connection to the node corresponding to the XID, correct?
> 
> In the meantime I'm doing more reading, but the example code doesn't quite appear to be an accurate analog of our environment.
In terms of these questions:
1. It shouldn't really impact things, except to say that access to *obtaining a reference or creating a new instance* on the subordinate transaction should be synchronized (in the example this is getAndResumeTransaction). This use case does add further legitimacy to registering the proxy *after* you return from the remote call. 
Jonathan may have a different perspective on this though? But as I understood it multiple threads accessing the same transaction is fine in JBoss TS.
2. Yes you are broadly correct, I do actually demonstrate this in the example. Take a look at the operations (and usages of) LocalServer::storeRootTransaction LocalServer::removeRootTransaction and LocalServer::getAndResumeTransaction.
3.There are two things that need persisting, one is the information you indicated, you can see that this must be done in two places, once when you talk to the remote server (before enlisting an actual proxy) and once again when the proxy is enlisted and you have a real Xid - if we did use your approach above we would only need to do this once as we can calculate the proxies XID before it is actually enlisted. In the example this is done when the proxy is created (ServerImpl::generateProxyXAResource) and updated when the proxy is enlisted (I delayed it to ProxyXAResource::prepare, this is the persist that could be elimated). That covers your point 3 above, but you also need to persist the dynamic subordinate node identifier at the moment.

Based on the discussions so far, the most appropriate changes would appear to be to the XID, to make special ones for proxy resources and to key EIS names as int so we have room for subordinate names for normal Xids. Making these changes would:
1. Impact existing users from a usability perspective that are already using the EIS name and expecting to read this from the persisted Xid somehow.
2. Require investigation to determine if customizing bqual for a particular type of resource (saying implementing a ProxyXAResource interface or similar) was even possible
3. Still require the transport to make an additional log write per transaction the list of servers it talks to in order to recover transactions at them.

That said, I am due to go on paternity any time now for two weeks so would be reluctant to alter the code too much further now (e.g. keying the EIS name or bespoke XID creation) but I can certainly investigate both of those in a branch.
--------------------------------------------------------------

Reply to this message by going to Community
[http://community.jboss.org/message/633431#633431]

Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2041]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-dev-forums/attachments/20111025/b4426d71/attachment.html