7:25 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632161#632161
--------------------------------------------------------------
Hi,
The implementation for this issue (registered as https://issues.jboss.org/browse/JBTM-916
https://issues.jboss.org/browse/JBTM-916) has been put back to svn.
The implementation I went for structures the XID as follows:
int formatId = // new sequential number
byte[] globalTransactionId = // Uid + Node name of transaction originator
byte[] branchQualifier = // Uid + Node name of subordinate + Node name of subordinates
parent + EIS name
In order to fit the node names in the space available they have had to be encoded as
integers (rather than strings as they used to be). I appreciate from Davids points above
that this will add configuration complexity as nodes must be assigned a unique transaction
manager identifier and remoting string but at the moment I can't really see any other
way around this (I will elaborate further below on why the data is required).
I am happy for the node names to be changed back to Strings, but we have a space issue
then which would definitely need addressing.
The space we are most constrained by is the bqual, it must contain a Uid is 28 bytes (this
guarantees we have a globally unique identifier for any particular branch), it also needs
two node identifiers which I opted for using integers (i.e. 2 x 4) , and an EIS name that
can consume the remaining part of the bqual and is typically the JNDI name of the resource
which can be up to 28 bytes. If there was someway to compress three reasonable length
strings into the bqual (or put another way we can be certain that two string node names
and an EIS resource name can be encoded into javax.transaction.xa.Xid.MAXBQUALSIZE -
com.arjuna.ats.arjuna.common.Uid.UID_SIZE bytes [64 -28]) then it is a trivial change to
revert node names back to Strings.
Now, onto the reasoning for why the data is required at each point, before this I will
clarify one piece of terminology:
orphan resource = a resource that has prepared and stored an XID but where the transaction
manager did not have chance to write out a record of the XAResource before it crashed
A> global transaction ID - This definitely needs a UID for uniqueness, it also needs
the node name encoding in it so that orphaned (JTA) resources can be rolled back.
B> branch qualifier - Each RM needs a identifier unique within the global transaction
ID, it also needs the subordinate node name to ensure the transport is able to filter out
Xids returned from the XATerminator recover method so that different root transaction
managers don't try to recover each others transactions. It also needs a subordinates
parent node name for a similar reasoning to the node name encoded into the gtrid, i.e. to
ensure that when a recovery manager for the suboridnate starts up it can detect XA
resources of a subordinate transaction where the XAResource prepared and stored the Xid
but the subordinate transaction itself failed to prepare.
In terms of the bqual, it is not an absolutely vital component to store the parent node
name. If you look at
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
-r 37583 you can see that as I have a single ProxyXAResource per Xid and not a single XAR
for all Xids I am able to filter the Xids returned from XATerminator.recover on format ID
and gtrid alone. This is because I know that per subordinate transaction there should only
ever be one single proxy XA resource talking to it (see the design discussion I am about
to create for more details on this). Providing the parent node name though allows you to
create a proxy XA resource that can recover all XIDs for a subordinate in one go, rather
than calling recover multiple times.
Hopefully this clarifies the motivation why the Xid has been encoded as it has and why I
have had to move to integers for node names.
Tom
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632161#632161]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
9:34 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632280#632280
--------------------------------------------------------------
Have you tested an performance implications of this?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632280#632280]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
9:47 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632284#632284
--------------------------------------------------------------
Hi Mark,
In terms of raw XID manipulation:
I have ensured that in the case where we have none-distributed JTA the cost of this is
zero (well, an initial check when generating the first XID has to compare a boolean to see
if this is a subordinate).
In the case of existing JCA code there are a couple of further boolean checks when
resolving existing SubordinateTransactions to check if the inflowed XID formatID is equal
to one of ours, if false, do old comparison, if true do new comparison (which actually
scores a hit by comparing gtrid only so that should be somewhat faster than comparing
gtrid and bqual).
So essentially, the current system should remain as performant as it was. The distributed
JTA case I have not performance tested as it is new functionality with nothing to compare
against, though as the node identifiers are now integers I suspect encoding an XID will be
more efficient than encoding an older XID.
In terms of architecural principal:
My tests for this simulate distribution via classloader separation. When David is able to
consume the TS work, we can see what the cost of the distributed JTA vs JTS is "in
real life". That said, I have placed a restriction that the graph of proxy XA
resources must not contain loops (working with this is demonstrated in my example). This
will ensure that transaction completion is efficient as you don't loop around the same
nodes during completion, e.g. a transaction that flows through nodes 1,2,3,3,1,3,1,3 would
get automatically merged (as the transport knows not to add completion links for nodes
that are already connected in this transaction) and therefore gets ordered as 1,2,3
(though unfortunately not 1,2 and 1,3 which would be more efficient, but as the link from
2,3 is created before the link from 1,3 there is nothing we can do).
Tom
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632284#632284]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
7:23 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632474#632474
--------------------------------------------------------------
Have you run performance tests or is this on paper only?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632474#632474]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
8:02 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632479#632479
--------------------------------------------------------------
Performance tests on the XID generation?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632479#632479]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632509#632509
--------------------------------------------------------------
That'd be the place to start :)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632509#632509]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:09 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632523#632523
--------------------------------------------------------------
Sure, but like I mentioned above. I haven't really altered the XID creation too much.
Basically, the old structure is:
XidImple(OLD)
int formatId
byte[] gtrid { UID + nodeName:String }
byte[] bqual { UID + eisName:String }
XidImple(NEW)
int formatId
byte[] gtrid { UID + nodeName:int }
byte[] bqual { UID + nodeName:int + nodeName:int + eisName:String}
Generating the two should be fairly similar shouldn't it? Or maybe I am missing
something? To create a harness to test this would require both XidImple to be present
(which as the new one replaced the old one would mean two builds of TS).
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632523#632523]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:18 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632527#632527
--------------------------------------------------------------
Well I'm assuming there will be zero impact on performance (or maybe slightly better
performance).
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632527#632527]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:22 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632528#632528
--------------------------------------------------------------
Same here :)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632528#632528]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:25 p.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632529#632529
--------------------------------------------------------------
Are you really, absolutely certain that there's no other way to solve this than to
assign int IDs to every node? As I've said this is going to be a very tough sell - at
least with unique names, we can generate them based on the system host name and the domain
configuration. With int IDs, the space is too small for a full identifier; it's even
too small for something like a cryptographic key or hash which is at least *likely* to be
distinct.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632529#632529]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:44 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632535#632535
--------------------------------------------------------------
Unfortunately I am sure after having several discussions with Jonathan about this (though
happy to be disproved).
What we absolutely must be able to do is allow the recovery manager to detect orphan
resources.
Typically we detect the orphan by looking in the gtrid for the node name (recall it is UID
+ node name). And that is OK now and moving forward for the none distributed case.
In the distributed case we are adding though, when the recovery manager is looking for
orphaned resources registered against the *subordinate* it can't use the node name
from the gtrid as that is the node name of the root TM rather than this subordinate.
Therefore I have put the node name of the subordinate in the bqual (which the subordinate
transactions can edit).
Unfortunately a bqual can only be 64 bytes. In this 64 bytes we have to get:
1. The Uid (this could have been your path solution catted with a sequence number but I
went with Uid to minimise change but they are the same for this discussion as they are
only part of a bqual) - a Uid is 28 bytes constant
2. The "EIS name" - as I mentioned this is a requirement from customers - a
variable length string
3. The subordinate node name (so crash recovery can detect orphan XA resources for the
subordinate as described above)
4. The parent node name (this was added so the transport can choose to be implemented by a
single proxy XA resource per local server that can recover for multiple remote servers and
therefore needs to filter out the Xids it gets back from the call to XATerminator::recover
by parent node name rather than Xid) *potentially this can be removed - the example I
created
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
doesn't actually use this as I chose to create a new proxy per transaction,
performance testing of the transport will indicate which is the more efficient*
So at a minimum we need to get (i.e. if we confirmed that the parent node name would not
be of use):
1. The unique number for the branch
2. The EIS name
3. A node name
Into the 64 bytes of the bqual
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632535#632535]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
12:37 a.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632558#632558
--------------------------------------------------------------
Is it possible to play tricks with the EIS name? I suspect that N(EIS) <<<
N(nodes), so maybe we can require EIS names to be maintained in a configuration file and
index that by 2 bits in the bqual. OK, maybe 2 bits is too small, but you get my meaning
;)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632558#632558]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
12:47 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632563#632563
--------------------------------------------------------------
@Mark great point, though we could actually do the same for the node name couldn't we
I suppose? And that would mean that we wouldn't be impacting our current code for EIS
name in favour of this new functionality
The AS can have a file that says
"David's" prefered name TS node name
----------------------------------- --------------------
"foo" 1
"bar" 2
"baz" 3
Basically it works a bit like a bespoke form of hashing where the hash of the remoting
identifier is manually specified in some file, as you suggested above basically but for
node names rather than EIS names.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632563#632563]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632568#632568
--------------------------------------------------------------
To clarify, if we did knock the EIS name down to an int/short as Mark indicated was
feasible. We would have the following:
byte[]:64:bqual
{Uid:28:sequenceId,String:variableLength:subordinateNodeName,String:variableLength:parentNodeName,int/short:4/2:EISName}
Meaning we would still need to be able to get two strings into the remaining 34/32 bytes,
another reason I suggested converting the node name in the AS down to an int as that saves
the space of two strings rather than just one.
Plus the API for getting the EIS name is already out there and is being used (I assume).
We would therefore need to get folks to change their implementation of:
org.jboss.tm.XAResourceWrapper::getJndiName to return this integer key for the EIS name
table (or pay the cost of looking up this String in the EIS String to int table each time
and XID is created).
Either approach (keying node names or keying EIS names) levies an administration overhead
but for space considerations and the fact that the EIS name API is already out there I
think we should use ints in the XID for nodes rather than EIS names - again though, happy
to be dissuaded.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632568#632568]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
7:13 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632703#632703
--------------------------------------------------------------
I'd be more concerned about doing this with the node name than with the EIS, since the
former is not as dynamic as the latter, i.e., I suspect the EIS data is going to be almost
read only whereas the node names will be added to almost constantly. That'd make a
central repository or cache for node names far less useful in a failure scenario than the
equivalent for EIS.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632703#632703]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
7:29 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632705#632705
--------------------------------------------------------------
Hi guys,
I just want to prefix this answer with the statement that given unlimited data length I
don't mind whether or not node names were encoded as Strings, Ints, BLOBs or GLOBs.
The administration of this thing is one thing. But in terms of the XID structure it only
has 64 bytes to play with in the bqual.
The bqual needs the following:
1. Sequence number - we all agree with this
2. Parent node name - This is vital for recovering orphan subordinate transactions as
explained above
3. Subordinate node name - This is vital for recovering orphan subordinate resources as
explained above
4. EIS name - required for debug
Lets say we keyed the EIS name down to an int - 4 bytes - using Jonathans suggestion and
the algorithm I indicated earlier, thereby paying the performance cost of reading/writing
to a dynamic EIS key file and looking up the EIS name in the key map each time.
Next, the sequence number is 28 bytes because it is a Uid, we could look at trying to make
this smaller but it needs to be unique within the scope of the gtrid.
That leaves us 32 bytes to encode the two node names (parent and subordinate). If someone
can confirm that we can have a validator on CoreEnvironmentBean that says node names that
can't be .getBytes("US-ASCII").length'd down to 12 bytes or less can be
"throw new CoreEnvironmentBeanException"'d is fine, I can move back to
Strings for these within an hour or so (I say 12 bytes so we can write the byte[] and its
length - an extra 4 bytes, to be fair we could use a short here so call it 14 bytes as the
max getBytes.length of the node name).
I would also need it confirming that we are fine to make the adjustment of the EIS name
down to an int which is just logged to file and so admins debugging issues will need to
crack open that file to reverse lookup the actual EIS name from the keys which are going
to be encoded in the Xids.
Tom
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632705#632705]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
8:57 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632733#632733
--------------------------------------------------------------
I know how long an XID struct is ;)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632733#632733]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
9:12 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632736#632736
--------------------------------------------------------------
Hi Mark - sorry, I know you do!! Just setting out the issue with node names being stored
uncompressed, unless they can be encoded into 14 bytes is all. If the node name can't
be fit into 14 bytes as a string then it is going to need to be indexed somewhere and the
only thing I can think of is the central file with node names in.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632736#632736]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:34 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632761#632761
--------------------------------------------------------------
I'm joking with you :)
Can you compress the sequence number? What I mean by this is that if it's a Uid and
based on the gtrid then maybe you can get rid of the IP address component of the sequence
number because it's implicitly there in the gtrid anyway?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632761#632761]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:41 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632764#632764
--------------------------------------------------------------
Yeah, possibly, I wonder how much space we are going to save, but yeah, I can take a look
at that. That said, what size should we be looking at aiming for allowing node names? At
the moment, it is possible that 14 bytes is enough to encode the node name? David, what
should be aiming to allow size wise for a remoting node name?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632764#632764]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:49 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632768#632768
--------------------------------------------------------------
Well if you use the node name bytes as bits and assuming a number is sufficient for the
name and not a string, then you've got 112 bits to work with which allows you to
encode about 5.2E33 numbers. I doubt anyone will have that many nodes. So maybe
there's scope there too?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632768#632768]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:55 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632770#632770
--------------------------------------------------------------
I doubt that they would have more than that too - although it would make an interesting
case study :)
But I also assumed that no one would have more unique nodes than would fit in a standard
int either. I think the problem with encoding "node names" as numbers is that
David is saying he doesn't think people will want to administer the nodes as numbers,
rather he preferes a string derived from the hostname and domain (out of interest David
how do you propose to deal with multiple nodes on the same machine, plus remember that for
TS these node names are sticky per instance you want to recover).
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632770#632770]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:15 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632775#632775
--------------------------------------------------------------
Aren't we mixing two different issues here? How we represent the node name within TS
(within the XID structure) and how that node name is presented to users/administrators?
Our top priority has to be data consistency in the presence of failures. If that means
that a sys admin either can't do something for a while (because, say, a centralised
name server that maps int node names to a nice string is unavailable) or has to work with
ints, then I'm prepared to consider that any day over the alternative.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632775#632775]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:21 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632776#632776
--------------------------------------------------------------
Mark Little wrote:
Aren't we mixing two different issues here? How we represent the
node name within TS (within the XID structure) and how that node name is presented to
users/administrators? Our top priority has to be data consistency in the presence of
failures. If that means that a sys admin either can't do something for a while
(because, say, a centralised name server that maps int node names to a nice string is
unavailable) or has to work with ints, then I'm prepared to consider that any day over
the alternative.
+1
All I want is for nodeIdentifiers to be provided to TS through the CoreEnvironmentBean as
an integer and that the same recoverable instance of TS gets the same integer node name
each time, I can then happily encode this in the XID and failure recovery etc proceeds as
normal. What systems are used to provide that node identifier to TS should be largely a
black box to me.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632776#632776]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
7:22 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632965#632965
--------------------------------------------------------------
Mark Little wrote:
Can you compress the sequence number? What I mean by this is that if it's a Uid and
based on the gtrid then maybe you can get rid of the IP address component of the sequence
number because it's implicitly there in the gtrid anyway?
Sorry I didn't get chance to respond to this yesterday. I have created a BranchId
class which is a clone of UID but doesn't utilise the hostname part of it. I have run
a build now to check it hasn't broken anything and it is fine but I haven't put
this back until we know the route we are going to take.
Removing the hostname from BranchId and giving EIS names transient integer representations
gives us a space of:
64 (bqual)
- 12 (branch id)
- 4 - eisname key
= 58 bytes (available for node names)
That means there are up-to 29 bytes per name, we will need to store the name length (which
given the size available need only be a short) so less 2 bytes leaves 27 bytes to store a
node name.
If this is enough space (and Mark and Jonathan agree that this change to branch qualifiers
is sound - it will affect all Xids) then I can have this changed within an hour and we
can proceed to get this out into AS7 trunk. I will have to change the EIS name to an int
too of course, but I can add a routine that converts EIS names to the next available int
(or short if you shout up now) and logs out the conversion values to the log file as a
warning. Would this map of EIS names need to be persisted between runs? I assume not as
the transactions jboss logging output file will contain the id of the EIS against the
provided name for that particular run. If we *do* need to persist this mapping file then
we will need to have a think about the best place to put it, can we assume to dump it out
into the current working directory?
The alternative is to keep with ints for node name, and I can put this out there right now
with no change required to EIS name.
Either way I would like to get some agreement as this discussion hasn't slowed the
process down so far as I have been comprehensively testing this functionality relying
purely on the availablity of the node name "tokens" in an Xid but as I near the
end of available tests I can perform, it really needs to get into Davids hands to provide
this new functionality into AS7.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632965#632965]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
8:55 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632987#632987
--------------------------------------------------------------
Let's get a response from David before proceeding, but I continue to believe that a
number is fine as long as we can provide some higher level tool to isolate users from
this.
BTW, if it's 27 bytes then does that mean it could (almost) be a Uid?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632987#632987]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
9:15 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632992#632992
--------------------------------------------------------------
Unfortunatly my brain failed me on my previous comment :s
64 - 4 - 12 is 48 not 58 :(
This leaves 24 (I hope!) bytes per node name (less a variable length short key of 2).
Therefore David, if you can confirm Integers are OK with you then we can start the ball
rolling on getting this to AS7 for you and I can show you the finalised example I have
been working on that should prove very helpful when implementing the transport
interceptors...
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632992#632992]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:42 p.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633034#633034
--------------------------------------------------------------
If I understand correctly, you're using integer values internally but the user
doesn't have to be aware of it? If so I'm in favor. Can you enumerate what
additional configuration items the user will require with this solution?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633034#633034]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
12:21 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633049#633049
--------------------------------------------------------------
Sorry, its not quite as you say. The user has to configure various items for the
transaction server and most of those remain unchanged. The one change that must be made is
that the CoreEnvironmentBean.setNodeIdentifier () must be changed from a String to an
Integer (and the coresponding JTAEnvironmentBean.setXARecoveryNodes).
This is app server configuration and should be unique for nodes sharing an object store
(or sharing a remoting domain in the new scenario)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633049#633049]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:31 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633073#633073
--------------------------------------------------------------
OK, unfortunately it isn't acceptable to require the user to configure int values like
this. But... what about this idea?
Basically you're looking for approximately 32 bits of uniqueness right? Well a node
name is based on host name, which is ultimately based off of DNS labels. A DNS label can
contain one of 26 letters, one of 10 digits, a "-", or a "." (but
cannot start or end in "-" or "."). This means that a single
character has only 38 possible values, which means you can fit about 6 of them into 32
bits, if you squeeze really hard.
Perhaps we can set it up so that we have just a couple more bits - maybe 42, which would
be just enough for 8 characters - and default it to the compressed node name, and if the
node name is greater than 8 characters we can print a warning and suggest that the user
choose a unique 8-character name?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633073#633073]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
5:23 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633157#633157
--------------------------------------------------------------
Hi David,
I was wondering about something like that (i.e. reducing the "domain" of the
character space). The main reason I backed out is a point I raised earlier:
"node names must be unique per instance"
This means that the host name (or a derivative thereof) is not unique enough. Lets say you
have two instances on the same machine, lets say we got it into 42 bits (although to be
fair we could allocate 6 bytes per name if that helped - 28 for the Uid, 6 per node name
leaving 24 bytes for the EISname). Anyway lets say the name was "foo".
If you start up two servers with the node name "foo" you can't flow
transactions between them as the node identifier is the same, fairly clear if
"foo" is on a different server, but potentially it could be on the same server,
i.e. the user could start up more than one instance on host "foo". Hence you
would need to configure node identifiers to be "foo1" and "foo2"
anyway, leading to an integer and therefore leading to needing to configure this integer
somewhere (presumably the same place where the integer I was talking about would need to
be - a central configuration repository). You see the node identifier defines who
"owns" transaction logs (this is an exisiting requirement - not new for this
release). If you are thinking about some kind of "launch counter" where we could
launch servers with a locally incremented suffix (as I would be :) ) then that isn't
really good enough as the user would want to deterministically know the server they are
launching to know which transactions are going to be restored.
The other issue is that hostnames may be none roman alphabet leading to quite a few more
options, take the servers listed on:
http://en.wikipedia.org/wiki/List_of_Internet_top-level_domains#Test_TLDs
http://en.wikipedia.org/wiki/List_of_Internet_top-level_domains#Test_TLDs or the ones from
http://www.w3.org/2003/Talks/0425-duerst-idniri/slide12-0.html
http://www.w3.org/2003/Talks/0425-duerst-idniri/slide12-0.html such as
http://räksmörgås.josefsson.org/ http://räksmörgås.josefsson.org/
Also, arguably by asking the user to "choose a unique 8-character name" we might
as well ask them to choose a unique int instead?
Just to clarify, this requirement for unique node names is an exisiting requirement of
JBoss Transactions, but in the JTA none-shared object store scenario it has been
sufficient to give the same node identifier.
Tom
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633157#633157]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
3:49 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633172#633172
--------------------------------------------------------------
I understand, it isn't perfect, but we do already have the requirement that our
(hostname-based) node names be unique for EJB, management, clustering, etc. to function
correctly. When more than one server is started up on the same host via domain
management, the node name is based on the host name plus the server name (which is unique
within a server group, which is a very manageable namespace as it is centrally defined).
Basically if we start up two servers with the same node name, there are a whole host of
services that won't function correctly if the user makes the servers interact.
It's still a lesser burden to choose a name which can mostly be defaulted to the node
name which already must be unique than to choose an integer. Otherwise we're back to
the drawing board to come up with a scheme which can use either a full name or topology
information (or some other scheme, like topology + full name hash) to fill in the
knowledge gap that each node needs for recovery. Using a plain int simply is not an
option.
As for the non-roman alphabet-based names, they encode back to the same character set as
any other domain name, so we don't really need any special handling for this case as
we can perform the same encoding. It would be noted however that such names would have a
significantly reduced maximum length, potentially as few as 2 or 3 characters.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633172#633172]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
12:16 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633307#633307
--------------------------------------------------------------
Just to round off the discussion:
Basically, I determined that the subordinate node name is not strictly a "node
identifier" in the same sense as we are requiring it for a JTA gtrid value (which
must be unique per object store). Instead it need only be unique per propagated
transaction and the server does not need it to be the same value for different
transactions flowed through the it, nor indeed must the value be globally unique (i.e.
different servers can reuse the same id so long as it is not used by a different server
within the scope of the same transaction).
All that that does need to happen is that the transport dynamically assign a unique
integer for each node participating in a transaction, e.g. a counter starting at 1 and a
recoverable mapping file be stored *by the transport* so that for any given transaction a
server (keyed by the traditional String node identifier) knows what its own (now dynamic)
"subordinate node name" was for the transaction, thereby assisting it with
recovery.
To that end I have only had to change:
XID {
int:formatId
byte[]:gtrid { UID:sequence, String:nodeName }
byte[]:bqual {UID:sequence, String:EIS name }
}
To:
XID {
int:formatId
byte[]:gtrid { UID:sequence, String:nodeName }
byte[]:bqual {UID:sequence, int:subordinateNodeName, String:EIS name }
}
(i.e. added that single integer into the bqual).
It does mean the transport needs to work a little harder to maintain this mapping and
dynamically track available subordinate node names for each transaction flow but has
solved the issue with dealing with dynamically allocated subordinate node names. It also
means that setNodeIdentifier is totally compatible with previous versions of AS (i.e. any
String is allowed for a node identifier, not just an Integer encoded as a String) and has
therefore further minimized the changeset required from the core JTA side so is safer from
an existing functionality point of view.
Thanks to everyone for their suggestions on this, also, clearly the restriction about the
(now reverted to String) node identifier being unique within the domain is all the more
more vital.
For those tracking this discussion, the reason the parent node name is no longer required
is because: a> only one proxy can be added for any particular transaction at any
particular server and b> the proxy xa resource transport addition must track
transactions in flight for each server so therefore it takes the place of this particular
value
For those who are wondering about the subordinate node name is now able to be dynamic, the
reason is the we rely on the transport must storing this data now rather than reading it
from one of our configuration files.
Essentially the requirements of the solution are the same as they have always been, it is
just that a little bit more data is required to be stored by the transport than was the
case before.
I do have examples and tests for all of this:
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
and
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin....
In particular, the example should prove to be extremely useful when developing the
transport required for this as it shows and describes the points where relevant
transaction data must be reliably persisted.
As I say, by reverting to a String node identifier, the solution is even closer to the
original mechanism so therefore this new approach should have no impact on existing
JTA/JCA work. The only difference now is that the space available to EIS namesis one byte
shorter (from 36 to 35 bytes).
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633307#633307]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
12:28 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633314#633314
--------------------------------------------------------------
I should point out that the impact of moving from static subordinate node identifiers to
dynamic ones is that an extra log write/delete per transaction is required, which will
clearly have an impact on performance.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633314#633314]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:11 a.m.
Michael Musgrove [http://community.jboss.org/people/mmusgrov] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633322#633322
--------------------------------------------------------------
Couldn't the mapping be stored with the transaction log record thereby avoiding the
need for two separate writes. A further benefit would be the transport would be using our
optimised logging mechanisms.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633322#633322]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:21 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633347#633347
--------------------------------------------------------------
Hi Mike,
Hopefully my previous answer helps to clarify some of the motivations for the current
writes. It should also be pointed out that I don't mandate how this data is persisted,
indeed potentially there are some items that the transport must store which can be
persisted using TS optimised logging mechanisms, that is left to the transport to
determine.
Thinking about your suggestion, one example where TS logging could possibly help to
optimize this is the dynamic subordinate node identifier. As this Xid is for one of our
own transactions, potentially David could provide his own XidImple implementation which
packs the static subordinate node and its parents node identifier name after the bqual
(which would be String node identifier instead of using a transport allocated dynamic int
as I was describing earlier) in as an extra attributes to be persisted. Although it is
likely that doing so would require work throughout the code base to ensure it was feasible
to allow. Also, for example, I assume the JDBC action store assumes a schema that can take
a BLOB(128) for the data part of an XidImple we would therefore need to determine which
objectstores this would work with.
NOTE: This is a sponateous response to your point, I am not entirely certain it would
work!
If you didn't go down that route (or it is proved not to work) then the issue you have
is the transport needs to persist items at a different time to that which the transaction
service current does. As Jonathan has pointed out, the transaction service is heavily
optimized in this regard and with the particular set of requirements in hand, it seems
that we have to add a few of these persistence points back (well the transport adds them,
specific for this feature).
I will think more about providing a bespoke XidImple that packs in more data than a Xid
typically allows - thanks for the suggestion :)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633347#633347]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:05 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633341#633341
--------------------------------------------------------------
I just had an interesting chat over on our projects freenode irc channel #jbossts where
some legitimate concerns around performance of the solution were discussed. I thought it
might be useful to reiterate some of the points I tried to address there for the rest of
the community.
Persistence points of the solution:
1. The transport is required to maintain a persistent store of all Xids that it proxies to
the remote side, it has two points where it needs to manipulate this, first during the
initial contact with the remote server we must record at least the fact that we have
talked to a remote server for a particular gtrid. Once we have access to the proxies
actual bqual we need to update this record to ensure that the proxy can be recovered
correctly in the case of a viable commit - in the current example I delayed this to
prepare. We need to do this as we cannot filter the remote server's XIDs as XIDs do
not have the parent node name encoded in them.
2. When a transaction is first propagated into a new server we must persist the dynamic
subordinate node identifier for this server so that we can reliably recover transactions
for this particular instance of server.
Removing these addtional log writes is feasible, but would require the change to allocate
static node identifiers that can be compressed into the bqual for each node.
Removing the log writes associated with point 2 above can be done by using the nodes
existing node identifier and compressing this into the bqual. This should be a relatively
painless operation as we could use the earlier suggestion around compressing the EIS name
into the integer, leaving ample space to encode a single node identifier in the bqual.
The bqual would move from byte[]:bqual { UID:sequence, int:subordinateIdentifier,
String:EISname } to byte[]:bqual { UID:sequence, String:subordinatesNodeName, int:EISname
}. It would require minor changes as explained earlier to ensure that the converted EIS
name was logged in much the same way I outlined logging the subordinate name but is fairly
trivial and is essentially compatible with the current approach.
Removing the log writes associated with 1 is slightly more involved. It is this scenario
that necessitated the introduction of the parent node name. In this scenario the server is
solely responsible for keeping a record of all transactions it has seen - as it does
anyway - but this list can then be filtered by parent node name in recovery. Furthermore,
to remove the requirement of keeping a record of the XIDs the proxy has seen you would
also need to propagate a list of all nodes (remoting transport node name is fine) that a
transaction has flowed through to ensure that you can register a proxy *before* you
propagate to the remote server. The alternative is to always put a proxy in place but
beware of the scenario where you have created a diamond, e.g. a flow going 1,2,3,1,3 you
would get two "recover/completions" called on node 3 in that case, the second
recover/completion sequence causing an error as the subordinates XID is already recovered.
Yet another alternative to remove one of the log writes related to point 1 above would be
for proxy xa resources to have a deterministic branch IDs, but we would still need the
parent node name attribute adding to the XID.
Hope that clarifies some of the investigations I have done. The final approach was driven
mainly by two key factors:
1. Ensure that the performance of existing JTA/JCA is not impacted
2. The servers node identifier must be encoded as a String
Unfortunately there are performance impacts on the new functionality as a result but at
least we have a basis for further discussion when we come back to review this.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633341#633341]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:48 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633485#633485
--------------------------------------------------------------
This concerns me greatly. We've spent years optimising TS. Presumed abort semantics
help here obviously, but there's been a lot of other work. Let's not negate that!
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633485#633485]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:23 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633490#633490
--------------------------------------------------------------
Hi Mark,
The main issue with delivering this functionality is the additional 3 persistence points,
2 of which we have covered by providing an alternative XID implementation for proxies and
subordinates, 1 of which is optional anyway.
1. The optional persistence point: This persistence point seems to remain, though still
optional: when recovering, you need to know which servers to talk to to call recover on
for unprepared transactions, you could argue that transaction timeout is enough for this
and it then becomes an transport implementors prerogative whether to persist this
information (perhaps this is only done if the transport detects that the transaction does
not have a timeout).
2. Using alternative XID implementations for proxy XA resources (which have the
subordinate and parent node name in the bqual) and for subordinate transactions (which
needs subordinate, parent and parents parent) should remove the requirement to record XIDs
at a caller server. The reason that the subordinate needs three node names is because the
proxy xa resource is enlisted at node 2 say with with a bqual (2,1), it then needs to ask
the remote server (say 3) for all subordinate transaction XIDs that the server it is
running in as a parent are part of . It will then need to convert these Xids to one its
own local server understands was part of a transaction (by dropping the recovered Xids
subordinate node name and bumping the other two down a notch - i.e. converting a bqual of
3,2,1 to 2,1). Unfortunately, the bqual of the subordinate JTA transactions will have a
bqual length over 64 bytes, so this does still need investigation to see how different
objectstores cope with this.
3. The normal XID will need to be changed to have the String node identifier used in the
bqual and alongside a shortened EIS name, this will get rid of the requirement for dynamic
subordinate node allocation, it is relatively painless but will need Jonathans buy in as
it will be impacting funtionality (EIS name) that he has provided.
Of course, in reality I have parked this till after I get back from paternity ;)
Tom
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633490#633490]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:02 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/634251#634251
--------------------------------------------------------------
Just to confirm, the work related to compressing the EIS name down to an int by storing a
mapping file is actually done and in the branch as it is work not strictly related to XID
changes for distributed JTA but is a functional requirement so it made sense to get it in
the next release.
Therefore, when I get back from paternity the following changes may be made:
1. We can get rid of the requirement for the transport to maintain the subordinate node
name as a dynamic integer as there is now enough space in the bqual to put the node name
as a string (assuming 64(bqual length) - 28(Uid) - 4(eis key)) 32 bytes is enough to store
a remoting name. Of course, if we ever need to put something else in the bqual that will
not be possible again.
2. I can see a place where it would be relatively straighforward to add the parents parent
node identifier (jca.SubordinateAtomicAction). I will probably create a different class
for this (distributedjta.SubordinateAtomicAction), this gets rid ofone of the other
persistence points of the transport.
Both of these changes appear relatively trivial to make again but I wont destabilise
branch before I leave (hopefully I wont destabilise it on return either).
I believe the transport will still need the initial persistence point, that is it needs to
know (for outstanding transactions) which servers it has talked to so that it can call
recover on them in the case of failure. This is optional and we can rely on timeout
typically, unless timeout is not set...
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/634251#634251]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:19 p.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633411#633411
--------------------------------------------------------------
Tom Jenkinson wrote:
Just to round off the discussion:
Basically, I determined that the subordinate node name is not strictly a "node
identifier" in the same sense as we are requiring it for a JTA gtrid value (which
must be unique per object store). Instead it need only be unique per propagated
transaction and the server does not need it to be the same value for different
transactions flowed through the it, nor indeed must the value be globally unique (i.e.
different servers can reuse the same id so long as it is not used by a different server
within the scope of the same transaction).
All that that does need to happen is that the transport dynamically assign a unique
integer for each node participating in a transaction, e.g. a counter starting at 1 and a
recoverable mapping file be stored *by the transport* so that for any given transaction a
server (keyed by the traditional String node identifier) knows what its own (now dynamic)
"subordinate node name" was for the transaction, thereby assisting it with
recovery.
To that end I have only had to change:
XID {
int:formatId
byte[]:gtrid { UID:sequence, String:nodeName }
byte[]:bqual {UID:sequence, String:EIS name }
}
To:
XID {
int:formatId
byte[]:gtrid { UID:sequence, String:nodeName }
byte[]:bqual {UID:sequence, int:subordinateNodeName, String:EIS name }
}
(i.e. added that single integer into the bqual).
OK here's my first wave of
dumb questions:
1. Why do you need a UID in the bqual?
2. EIS name in this case is the local node's unique name for the subordinate it's
planning on talking to, right?
3. If so why do we also need subordinateNodeName? Just to create a way to differentiate
between the "first" inflow versus subsequent?
Tom Jenkinson wrote:
I do have examples and tests for all of this:
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
and
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin....
In particular, the example should prove to be extremely useful when developing the
transport required for this as it shows and describes the points where relevant
transaction data must be reliably persisted.
As I say, by reverting to a String node identifier, the solution is even closer to the
original mechanism so therefore this new approach should have no impact on existing
JTA/JCA work. The only difference now is that the space available to EIS namesis one byte
shorter (from 36 to 35 bytes).
OK I'm trying to make sense of this code now
w.r.t. our stuff. More dumb questions:
1. It really looks like you're "handing off" a transaction between nodes so
that only one has the transaction at a time, but in the real world many nodes in the graph
may be performing useful work at the same time (potentially even from multiple threads)
under the same (global) transaction. How does this impact this scheme (from a TM
perspective, not a transport perspective)?
2. You talk about not committing at the root node... should we be tracking the
transactions by gtid and detecting the circular flow at the transport layer? I assume
that if that is the case, we'd use the original XID instead of the new alias and
we'd treat commit requests of the alias XID as a "no-op" sort of thing.
3. It seems to me that we'd only have to persist the XID and enough information to
revive a connection to the node corresponding to the XID, correct?
In the meantime I'm doing more reading, but the example code doesn't quite appear
to be an accurate analog of our environment.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633411#633411]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
4:34 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633431#633431
--------------------------------------------------------------
Hi David,
Sorry the example doesn't map directly to your use case, I condensed it from the test
suite and it is really more guidance on what tasks are required from the transport so
hopefully it has proved at least useful in that regard. Indeed as a talking point for
questions at least it seems to have helped :)
NOTE: My answers are below but they are caveated by the statement that they are broadly
based on the current approach.
David Lloyd wrote:
> Tom Jenkinson wrote:
>
> Just to round off the discussion:
>
> Basically, I determined that the subordinate node name is not strictly a "node
identifier" in the same sense as we are requiring it for a JTA gtrid value (which
must be unique per object store). Instead it need only be unique per propagated
transaction and the server does not need it to be the same value for different
transactions flowed through the it, nor indeed must the value be globally unique (i.e.
different servers can reuse the same id so long as it is not used by a different server
within the scope of the same transaction).
>
> All that that does need to happen is that the transport dynamically assign a unique
integer for each node participating in a transaction, e.g. a counter starting at 1 and a
recoverable mapping file be stored *by the transport* so that for any given transaction a
server (keyed by the traditional String node identifier) knows what its own (now dynamic)
"subordinate node name" was for the transaction, thereby assisting it with
recovery.
>
> To that end I have only had to change:
>
> XID {
> int:formatId
> byte[]:gtrid { UID:sequence, String:nodeName }
> byte[]:bqual {UID:sequence, String:EIS name }
> }
>
> To:
> XID {
> int:formatId
> byte[]:gtrid { UID:sequence, String:nodeName }
> byte[]:bqual {UID:sequence, int:subordinateNodeName, String:EIS name }
> }
>
> (i.e. added that single integer into the bqual).
OK here's my first wave of dumb questions:
1. Why do you need a UID in the bqual?
2. EIS name in this case is the local node's unique name for the subordinate it's
planning on talking to, right?
3. If so why do we also need subordinateNodeName? Just to create a way to differentiate
between the "first" inflow versus subsequent?
Here is the answer to these questions:
Interesting suggestion! It is indeed possible that for this type of XA resource we could
look at creating a bespoke XID with the format:
XID {
int:formatId
byte[]:gtrid { UID:sequence, String:nodeName }
byte[]:bqual {String:subordinateNodeName, String:parentNodeName }
}
We know we can do that because we know that there is only going to be (by convention) a
single proxy to the remote server for this node, therefore the branch is unique without
the Uid.
TS is not really geared up to support generating XIDs differently per XA resource, but we
can probably change that!
Not to say we shouldn't do that, but I was thinking about this (well a close
derivative of it, based on ints again sorry - but in my mind they are both tokens) over
night though, there is still an issue with storing the node identifiers in the XID which
the transport has to work around by persisting them.
Basically, if you did what you are saying (which is a neat idea). The proxy XA resource
would be enlisted with the ID (assuming nodes named 1,2,3 etc - Strings though):
ProxyXAResource - bqual{"2","1"}
Remote subordinate transaction - bqual{"3", "2"}
That means in recovery you can't ask the remote server for a list of XIDs it knows
about because it will return {"3","2"} but the local server
doesn't know about 3,2, in this scenario it knows about 2,1.
Also, "normal" XIDs would need to be able to fit a String subordinate node name
in them (for orphan detection). Meaning that we would definitely need to make space for a
normal bqual:
byte[]:bqual {UID:sequence, String:EISname}
As Jonathan suggested we can basically do this by changing EISname to be an int leading
to:
byte[]:bqual {UID:sequence, String:subordinatenodename, int:EISnameKey}
David Lloyd wrote:
> Tom Jenkinson wrote:
>
> I do have examples and tests for all of this:
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
and
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin...
https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Fin....
>
> In particular, the example should prove to be extremely useful when developing the
transport required for this as it shows and describes the points where relevant
transaction data must be reliably persisted.
>
> As I say, by reverting to a String node identifier, the solution is even closer to
the original mechanism so therefore this new approach should have no impact on existing
JTA/JCA work. The only difference now is that the space available to EIS namesis one byte
shorter (from 36 to 35 bytes).
OK I'm trying to make sense of this code now w.r.t. our stuff. More dumb questions:
1. It really looks like you're "handing off" a transaction between nodes so
that only one has the transaction at a time, but in the real world many nodes in the graph
may be performing useful work at the same time (potentially even from multiple threads)
under the same (global) transaction. How does this impact this scheme (from a TM
perspective, not a transport perspective)?
2. You talk about not committing at the root node... should we be tracking the
transactions by gtid and detecting the circular flow at the transport layer? I assume
that if that is the case, we'd use the original XID instead of the new alias and
we'd treat commit requests of the alias XID as a "no-op" sort of thing.
3. It seems to me that we'd only have to persist the XID and enough information to
revive a connection to the node corresponding to the XID, correct?
In the meantime I'm doing more reading, but the example code doesn't quite appear
to be an accurate analog of our environment.
In terms of these questions:
1. It shouldn't really impact things, except to say that access to *obtaining a
reference or creating a new instance* on the subordinate transaction should be
synchronized (in the example this is getAndResumeTransaction). This use case does add
further legitimacy to registering the proxy *after* you return from the remote call.
Jonathan may have a different perspective on this though? But as I understood it multiple
threads accessing the same transaction is fine in JBoss TS.
2. Yes you are broadly correct, I do actually demonstrate this in the example. Take a look
at the operations (and usages of) LocalServer::storeRootTransaction
LocalServer::removeRootTransaction and LocalServer::getAndResumeTransaction.
3.There are two things that need persisting, one is the information you indicated, you can
see that this must be done in two places, once when you talk to the remote server (before
enlisting an actual proxy) and once again when the proxy is enlisted and you have a real
Xid - if we did use your approach above we would only need to do this once as we can
calculate the proxies XID before it is actually enlisted. In the example this is done when
the proxy is created (ServerImpl::generateProxyXAResource) and updated when the proxy is
enlisted (I delayed it to ProxyXAResource::prepare, this is the persist that could be
elimated). That covers your point 3 above, but you also need to persist the dynamic
subordinate node identifier at the moment.
Based on the discussions so far, the most appropriate changes would appear to be to the
XID, to make special ones for proxy resources and to key EIS names as int so we have room
for subordinate names for normal Xids. Making these changes would:
1. Impact existing users from a usability perspective that are already using the EIS name
and expecting to read this from the persisted Xid somehow.
2. Require investigation to determine if customizing bqual for a particular type of
resource (saying implementing a ProxyXAResource interface or similar) was even possible
3. Still require the transport to make an additional log write per transaction the list of
servers it talks to in order to recover transactions at them.
That said, I am due to go on paternity any time now for two weeks so would be reluctant to
alter the code too much further now (e.g. keying the EIS name or bespoke XID creation) but
I can certainly investigate both of those in a branch.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633431#633431]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:45 p.m.
Mark Little [http://community.jboss.org/people/marklittle] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633484#633484
--------------------------------------------------------------
Some quick notes and thoughts:
* yes, multiple threads in the same TX are fine.
* we can create different XID formats as long as we change the format id (check out how
JTS does this with different types of interposition).
* leave this until you come back from paternity - it's an EAP 6.1 thing.
* if we are getting into circularities then we're moving back to a general distributed
transaction protocol and we already agreed that that would require JTS.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633484#633484]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
10:55 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/633487#633487
--------------------------------------------------------------
Thanks for the clarification, and as for leaving this alone till after paternity, I am
very much happy to! (breathes sigh of relief).
Although the required TS code changes have been fairly minor thus far, working through all
the error conditions is quite an adventure and has led to an extensive test harness being
developed to ensure TS supports access in this specific manner.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/633487#633487]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:02 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632570#632570
--------------------------------------------------------------
Mark Little wrote:
Is it possible to play tricks with the EIS name? I suspect that N(EIS) <<<
N(nodes), so maybe we can require EIS names to be maintained in a configuration file and
index that by 2 bits in the bqual. OK, maybe 2 bits is too small, but you get my meaning
;)
Yeah that's a good idea. What we're looking for isn't so much the
node name as it is a reproducable identifier for the +connection+, right?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632570#632570]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:27 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632578#632578
--------------------------------------------------------------
Just to be clear, I am still not sold on the idea of touching EIS name as the API is
"in the wild". That said, and terminology aside, what we need is some way to
know at which entity an XID was generated so that the recovery manager can detect it is an
orphan. I have been calling these node names, as the exisiting terminology is
"nodeIdentifier". Remember though, these are transaction manager
"nodes", not nodes as in hosts. Potentially there can be more than one
transaction manager node per host and each of these will need to be given a unique
persistent identifier. This is an existing and established requirement of JBoss TS.
I am not sure if I am misintrepetting Mark or David here but I think Mark was suggesting
putting the EIS name to EIS XID key into a configuration file so that we can then fit in
the node name as a String as David would like. I was saying that is not great as either
the established API needs to change to pass us the key or we take a performance hit of
resolving the key in the configuraiton file. Plus at the moment we are preferably going to
put two node identifiers in the bqual.
In either scenario, the administrator is going to need to keep a mapping file. Either
remoting node identifier to transaction manager node name (my preference) or EIS name to
EIS int key.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632578#632578]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:34 a.m.
Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632581#632581
--------------------------------------------------------------
In either scenario, the administrator is going to need to keep a
mapping file. Either remoting node identifier to transaction manager node name (my
preference) or EIS name to EIS int key.
That's not quite right. EIS names are scoped to the node and the mapping can be
maintained programmatically by e.g. the JCA xa recovery plugin registration code. The
nodeIdentifier is potentially enterprise (or at least data centre) scoped and needs manual
coordination and maintenance or some centralized/hierarchic registry service to manage
uniqueness. Than again so does allocation of a uniq string node name in the first place.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632581#632581]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:48 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632584#632584
--------------------------------------------------------------
I can see wanting the node name for debugging purposes in the gtid (for the originator)
and bqual (for the branch node). We'd have to make sure that we truncate names which
are too long though as the limit for node names is currenty 255 characters (though we
could reduce it further, as the maximum length of a host name label is 63 (recommended
maximum length is 24)).
But what I'm not clear on is, why exactly do we need the other node names in the XID?
I think that come out of off-forum discussions and without that information I can't
really offer up any ideas. It seems like we should be able to take advantage of the
unique topography of this scheme to provide some of the information.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632584#632584]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:54 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632587#632587
--------------------------------------------------------------
To be clear, the EIS name is the only debugging information in the XID.
XID:
formatId
gtrid { sequence, root node name }
bqual { sequence, subordinate node name, parent node name, eis name }
sequences are vital naturally
root node name is vital to detect orphans at the root in recovery
subordinate node name is vital to detect orphans at subordinates
parent node name is optional to filter list of recovered XIDs at a remote server by the
caller
eis name is debugging
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632587#632587]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:10 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632594#632594
--------------------------------------------------------------
Tom Jenkinson wrote:
To be clear, the EIS name is the only debugging information in the XID.
XID:
formatId
gtrid { sequence, root node name }
bqual { sequence, subordinate node name, parent node name, eis name }
sequences are vital naturally
root node name is vital to detect orphans at the root in recovery
subordinate node name is vital to detect orphans at subordinates
parent node name is optional to filter list of recovered XIDs at a remote server by the
caller
eis name is debugging
Okay so here's where my confusion came in.
When I suggested that sequence idea, the thought was that root node + sequence is
basically equivalent to (subordinate node name + parent node name) in that any participant
should be able to tell whether a branch belonged to it or its descendants, and also should
be adequate for filtering recovered XIDs. But I'm not sure what you mean by
"orphans" exactly - aren't all branches normally recovered by the root? If
so, does the subordinate path actually matter?
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632594#632594]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:20 a.m.
Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632598#632598
--------------------------------------------------------------
But I'm not sure what you mean by "orphans" exactly -
aren't all branches normally recovered by the root? If so, does the subordinate path
actually matter?
An orphan is a prepared RM whose immediate parent has died without writing a log. Under
presumed abort it has got to be rolled back. Only the immediate parent can do that - NOT
the root (unless they are the same). The reason is simple: only the immediate parent can
recognize the orphan status, as it's a function of the absence of a log record at that
location. Also, in some deployments only the immediate parent may have the driver and
connection details to reach that orphan - they are not guaranteed to be available at the
root. Part of the reason that nodes need a uniq id is that encoding it into the Xid
allows them to recognize their own orphaned branches and likewise avoid tampering with
ones they don't own. You can actually eliminate the need for globally uniq id by
writing to the log twice - once before and once after preparing the RM, in which case you
don't get orphans any more. The cure is worse than the disease though - those log
writes to disk drop your performance to unacceptable levels.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632598#632598]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:37 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632606#632606
--------------------------------------------------------------
Ideally the transactions would be recovered by the root TM. But you can have failure
scenarios which orphan a subordinate.
HOWTO orphan a subordinate
We only write out information about a transaction after the prepare phase.
Proxy cascades the prepare to the subordinate
--- Crash at root node ---
You now have an orphan!
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632606#632606]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
5:50 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632631#632631
--------------------------------------------------------------
Okay so the point is to avoid recovering orphans until the parent comes back online, and
then letting the parent do it? Because the hierarchical sequence thing works for that too
(it's easy to tell when there's an orphan because every node knows exactly which
subordinates it "owns").
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632631#632631]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
5:56 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632632#632632
--------------------------------------------------------------
Hi David,
No, its not quite that, that scenario you describe does exist, but thats not the scenario
termed an orphan in this case.
An orphan is where a remote resource has a record of the transaction but the remote
transaction doesn't.
Sequence:
1. Enlist resources at TM1
2. Propagate to TM2
3. Enlist resources at TM2
4. Return control to TM1
5. Call commit at TM1
6. Proxy the commit to TM2
7. Prepare the resources at TM2
8. -- CRAAAASH AT TM2 --
Now, when TM2 recovers it wont have any record of the transaction at TM2 so the prepared
resources at TM2 are orphaned forever. Hence TM2 needs to be able to be able to detect
these orphans and roll them back (presumed abort).
Tom
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632632#632632]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
8:47 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632638#632638
--------------------------------------------------------------
Another way a subordinate can be orphaned is by one phase commit.
1. Enlist a single proxy resource at TM1
2. Propagate to TM2
3. Enlist two resources at TM2
4. Return to TM1
5. Commit TM1 thinks there is only one resource so does a one phase commit
6. Propagate to TM2
7. TM2 knows there are at least two resources here so prepares resources
8. Crash at TM2
The resources prepared at step 7 are now orphaned
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632638#632638]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
3:10 p.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632656#632656
--------------------------------------------------------------
Not being able to sleep, I was thinking about the following. Another way an orphan can be
left (this time a subordinate transaction rather than a subordinate transactions resource)
is:
1. Start transaction at TM1
2. Propagate to TM2
3. Enlist the database at TM2
3. Return to TM1 but before the proxy resource can be enlisted at TM1....
4. -- CRASH --
This scenario has a couple of issues salient to us.
1. The transport is going to need to keep a record *before it talks to the remote server*
of all remote servers it talks to so that it can generate proxy XAResourceRecoveries to
make sure when the server housing it crashed it did not leave any transactions in flight.
2. The case provides the absolute requirement of storing the parents node name in an Xid
as during recovery this is the only information we have to help identify which subordinate
transactions in flight are owned by a recovering parent
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632656#632656]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:51 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632586#632586
--------------------------------------------------------------
Jonathan Halliday wrote:
> In either scenario, the administrator is going to need to keep a mapping file.
Either remoting node identifier to transaction manager node name (my preference) or EIS
name to EIS int key.
That's not quite right. EIS names are scoped to the node and the mapping can be
maintained programmatically by e.g. the JCA xa recovery plugin registration code. The
nodeIdentifier is potentially enterprise (or at least data centre) scoped and needs manual
coordination and maintenance or some centralized/hierarchic registry service to manage
uniqueness. Than again so does allocation of a uniq string node name in the first place.
Agree with you here, the scope of the EIS name needs to be unique within the node, so it
can be shorter. I think I maybe wasn't 100% clear but what I mean for EIS name was
administrator would still need to keep a mapping of EIS JNDI name to EIS "short"
name which is what I meant by keeping a mapping file in that scenario. At the moment the
EIS name is read from the JCA configuration that says "jndi-data-source-name"
basically (typically). If we no longer use that value then basically an extra bit of
configuration (which I was terming mapping) must be held to say for XYZ data source it has
a JNDI name of "foo" and a XAResourceWrapper identifier of "bar" -
well, most likely a shorter version of foo ;)
Also agree with you when you say "the nodeIdentifier is potentially enterprise (or at
least data centre) scoped and needs manual coordination and maintenance or some
centralized/hierarchic registry service to manage uniqueness. Than again so does
allocation of a uniq string node name in the first place." Basically, what this says
is we should not use the complexity of managing the node name as an argument for/against
Strings or ints.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632586#632586]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
1:56 a.m.
Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632588#632588
--------------------------------------------------------------
for EIS name was administrator would still need to keep a mapping of
EIS JNDI name to EIS "short" name
No, the server can simply allocate and store a new uniq short EIS name for any long name
it has not seen before. Because the server has view over the entire scope, so that mapping
process can be automated. That's the key difference with nodeIdentifier - a server
does not have sufficient local information to allocate itself a globally uniq node id.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632588#632588]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:10 a.m.
Tom Jenkinson [http://community.jboss.org/people/tomjenkinson] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632593#632593
--------------------------------------------------------------
Ah, OK, I can see what you mean now.
I am a bit concerned about that. Lets say the implementation of the compress routine was
(I am going to basically compress each name to an int as it seems logical to do it this
way instead of based on the lenght of the provided name):
static int counter;
static Map <String,int> names;
String jndiName = xaResourceWrapper.getJndiName();
int xidEisName = -1;
synchronzied (names) {
xidEisName = names.get(jndiName);
if (xidEisName == null) {
xidEisName = counter++;
names.put(jndiName, xidEisName);
System.out.prinltln("For the purpose of this run JNDI name: " + jndiName +
" is " + xidEisName);
}
}
//write XID out using the xidEisName instead of what was provided
A> It would be a performance hit
B> We couldn't guarantee the name was the same between runs unless we used a file
instead of static Map <String,int> names which would be even more of a performance
hit to write this each time we get a new datasource presented to us.
Jonathan Halliday wrote:
> for EIS name was administrator would still need to keep a mapping of EIS JNDI name
to EIS "short" name
No, the server can simply allocate and store a new uniq short EIS name for any long name
it has not seen before. Because the server has view over the entire scope, so that mapping
process can be automated. That's the key difference with nodeIdentifier - a server
does not have sufficient local information to allocate itself a globally uniq node id.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632593#632593]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:05 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632591#632591
--------------------------------------------------------------
Tom Jenkinson wrote:
> Jonathan Halliday wrote:
>
> > In either scenario, the administrator is going to need to keep a mapping file.
Either remoting node identifier to transaction manager node name (my preference) or EIS
name to EIS int key.
>
> That's not quite right. EIS names are scoped to the node and the mapping can be
maintained programmatically by e.g. the JCA xa recovery plugin registration code. The
nodeIdentifier is potentially enterprise (or at least data centre) scoped and needs manual
coordination and maintenance or some centralized/hierarchic registry service to manage
uniqueness. Than again so does allocation of a uniq string node name in the first place.
Agree with you here, the scope of the EIS name needs to be unique within the node, so it
can be shorter. I think I maybe wasn't 100% clear but what I mean for EIS name was
administrator would still need to keep a mapping of EIS JNDI name to EIS "short"
name which is what I meant by keeping a mapping file in that scenario. At the moment the
EIS name is read from the JCA configuration that says "jndi-data-source-name"
basically (typically). If we no longer use that value then basically an extra bit of
configuration (which I was terming mapping) must be held to say for XYZ data source it has
a JNDI name of "foo" and a XAResourceWrapper identifier of "bar" -
well, most likely a shorter version of foo ;)
From the Remoting perspective, the EIS name would most likely just be the full node name
of the calling node. If I mix these two up in the future, this is why. :)
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632591#632591]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:12 a.m.
Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632595#632595
--------------------------------------------------------------
From the Remoting perspective, the EIS name would most likely just be
the full node name of the calling node
Nope, it's the name of the target node, not the originating node. If you have one
parent server talking to two subordinate servers, that's two distinct EIS names,
because the parent has to be able to tell the subordinates apart for recovery purposes,
just as it has to be able to tell apart e.g. the Oracle and MSSQL server dbs it's
talking to through the same mechanism. Which brings up back to the other list the parent
needs to maintain: a set of all the subordinates it's talked to. Well actually
it's the set of all subordantes with outstanding tx, but deleting items from the list
frequently is probably more trouble than its worth.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632595#632595]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:18 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632597#632597
--------------------------------------------------------------
Jonathan Halliday wrote:
> From the Remoting perspective, the EIS name would most likely just be the full node
name of the calling node
Nope, it's the name of the target node, not the originating node. If you have one
parent server talking to two subordinate servers, that's two distinct EIS names,
because the parent has to be able to tell the subordinates apart for recovery purposes,
just as it has to be able to tell apart e.g. the Oracle and MSSQL server dbs it's
talking to through the same mechanism. Which brings up back to the other list the parent
needs to maintain: a set of all the subordinates it's talked to. Well actually
it's the set of all subordantes with outstanding tx, but deleting items from the list
frequently is probably more trouble than its worth.
Ah, you're just looking at
it from the opposite end; however the point still stands that the EIS name is equal to the
node name. If I set up a Remoting connection into the EJB client, the node (and cluster)
name of the remote side is noted for various purposes. This also has the advantage that a
node can be contacted via a different configuration, socket, or protocol and still be
recognized which allows us to port over UserTransaction as well as stateful EJBs to new
connections. It should also help us with this issue.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632597#632597]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:22 a.m.
David Lloyd [http://community.jboss.org/people/dmlloyd] created the discussion
"Re: Remote txinflow: XID changes"
To view the discussion, visit: http://community.jboss.org/message/632600#632600
--------------------------------------------------------------
Ah I see I said it backwards, my bad,
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/632600#632600]
Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
4572
days inactive
4586
days old
jboss-dev-forums@lists.jboss.org
62 comments
5 participants
participants (5)
-
David Lloyd -
Jonathan Halliday
-
Mark Little
-
Michael Musgrove
-
Tom Jenkinson