I just had an interesting chat over on our projects freenode irc channel #jbossts where some legitimate concerns around performance of the solution were discussed. I thought it might be useful to reiterate some of the points I tried to address there for the rest of the community.
Persistence points of the solution:
1. The transport is required to maintain a persistent store of all Xids that it proxies to the remote side, it has two points where it needs to manipulate this, first during the initial contact with the remote server we must record at least the fact that we have talked to a remote server for a particular gtrid. Once we have access to the proxies actual bqual we need to update this record to ensure that the proxy can be recovered correctly in the case of a viable commit - in the current example I delayed this to prepare. We need to do this as we cannot filter the remote server's XIDs as XIDs do not have the parent node name encoded in them.
2. When a transaction is first propagated into a new server we must persist the dynamic subordinate node identifier for this server so that we can reliably recover transactions for this particular instance of server.
Removing these addtional log writes is feasible, but would require the change to allocate static node identifiers that can be compressed into the bqual for each node.
Removing the log writes associated with point 2 above can be done by using the nodes existing node identifier and compressing this into the bqual. This should be a relatively painless operation as we could use the earlier suggestion around compressing the EIS name into the integer, leaving ample space to encode a single node identifier in the bqual.
The bqual would move from byte[]:bqual { UID:sequence, int:subordinateIdentifier, String:EISname } to byte[]:bqual { UID:sequence, String:subordinatesNodeName, int:EISname }. It would require minor changes as explained earlier to ensure that the converted EIS name was logged in much the same way I outlined logging the subordinate name but is fairly trivial and is essentially compatible with the current approach.
Removing the log writes associated with 1 is slightly more involved. It is this scenario that necessitated the introduction of the parent node name. In this scenario the server is solely responsible for keeping a record of all transactions it has seen - as it does anyway - but this list can then be filtered by parent node name in recovery. Furthermore, to remove the requirement of keeping a record of the XIDs the proxy has seen you would also need to propagate a list of all nodes (remoting transport node name is fine) that a transaction has flowed through to ensure that you can register a proxy *before* you propagate to the remote server. The alternative is to always put a proxy in place but beware of the scenario where you have created a diamond, e.g. a flow going 1,2,3,1,3 you would get two "recover/completions" called on node 3 in that case, the second recover/completion sequence causing an error as the subordinates XID is already recovered. Yet another alternative to remove one of the log writes related to point 1 above would be for proxy xa resources to have a deterministic branch IDs, but we would still need the parent node name attribute adding to the XID.
Hope that clarifies some of the investigations I have done. The final approach was driven mainly by two key factors:
1. Ensure that the performance of existing JTA/JCA is not impacted
2. The servers node identifier must be encoded as a String
Unfortunately there are performance impacts on the new functionality as a result but at least we have a basis for further discussion when we come back to review this.