[ https://issues.jboss.org/browse/ISPN-5261?page=com.atlassian.jira.plugin.... ] Matej Čimbora commented on ISPN-5261: ------------------------------------- I came across a variation of the original problem described by this JIRA. The impact is slightly different - transaction is commited on TX originator (A) and one of the participating nodes (B), however it gets rollbacked on another node (C). When get(K) is invoked on A, B & C send response, however each of them sends a different 'version' of the entry, depending on success of the transaction on particular node. Scenario - 4 nodes - edg-perf01-04 1. Transaction is started on edg-perf01, K is updated within transaction {code} 13:55:14,641 TRACE [org.infinispan.transaction.xa.TransactionXaAdapter] (DefaultStressor-1) start called on tx GlobalTransaction:<edg-perf01-60360>:181323:local {code} 2. Transaction is successfully prepared on edg-perf02 & edg-perf03 3. edg-perf01 invokes a commit command, which is sent to 02 & 03, check messageId & seqno {code} 13:55:36,061 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-1) real_dests=[edg-perf02-25521, edg-perf03-30958] 13:55:36,063 TRACE [org.jgroups.protocols.TCP] (DefaultStressor-1) edg-perf01-60360: sending msg to null, src=edg-perf01-60360, headers are RequestCorrelator: id=200, type=REQ, id=977458, rsp_expected=true, exclusion_list=[edg-perf01-60360], NAKACK2: [MSG, seqno=82297], 20903: slaveIndex=0, TCP: [cluster_name=default] {code} 4. On edg-perf02, the message is received & delivered. Commit is performed & entry is stored in DataContainer {code} 13:55:36,086 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (OOB-243,edg-perf02-25521) edg-perf02-25521: received edg-perf01-60360#82297 13:55:36,086 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (OOB-243,edg-perf02-25521) edg-perf02-25521: delivering edg-perf01-60360#82297 ... 13:55:36,148 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-86) About to send back response SuccessfulResponse{responseValue=null} for command CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:remote, cacheName='testCache', topologyId=228} {code} 5. As edg-perf03 received a new view, which does not contain TX originator, transaction gets rollbacked & removed from transaction table {code} 13:55:36,018 DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (OOB-2643,edg-perf03-30958) New view accepted: [edg-perf03-30958|70] (1) [edg-perf03-30958] ... 13:55:36,774 DEBUG [org.infinispan.transaction.TransactionTable] (transport-thread-24) Rolling back transaction GlobalTransaction:<edg-perf01-60360>:181323:remote because originator edg-perf01-60360 left the cluster {code} edg-perf03 then receives a new view, requests retransmission & edg-perf01 sends the commit once again {code} 13:55:38,719 DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,edg-perf03-30958) New view accepted: MergeView::[edg-perf02-25521|71] (4) [edg-perf02-25521, edg-perf03-30958, edg-perf01-60360, edg-perf04-55740], 2 subgroups: [edg-perf02-25521|69] (3) [edg-perf02-25521, edg-perf03-30958, edg-perf01-60360], [edg-perf04-55740|68] (1) [edg-perf04-55740] ... 13:55:40,798 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (Timer-3,edg-perf03-30958) edg-perf03-30958: sending XMIT_REQ ((7): {82296-82302}) to edg-perf01-60360 13:55:40,811 TRACE [org.jgroups.protocols.TCP] (Timer-3,edg-perf03-30958) edg-perf03-30958: sending msg to edg-perf01-60360, src=edg-perf03-30958, headers are NAKACK2: [XMIT_REQ, sender=edg-perf01-60360], 20903: slaveIndex=2, TCP: [cluster_name=default] 13:55:41,055 TRACE [org.jgroups.protocols.TCP] (INT-428,edg-perf03-30958) edg-perf03-30958: received [dst: edg-perf03-30958, src: edg-perf01-60360 (4 headers), size=64 bytes, flags=OOB|DONT_BUNDLE|NO_TOTAL_ORDER|INTERNAL], headers are RequestCorrelator: id=200, type=REQ, id=977458, rsp_expected=true, exclusion_list=[edg-perf01-60360], NAKACK2: [XMIT_RSP, seqno=82297], 20903: slaveIndex=0, TCP: [cluster_name=default] 13:55:41,081 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (INT-428,edg-perf03-30958) edg-perf03-30958: received edg-perf01-60360#82297 13:55:41,081 TRACE [org.jgroups.protocols.pbcast.NAKACK2] (INT-428,edg-perf03-30958) edg-perf03-30958: delivering edg-perf01-60360#82297 {code} Transaction is not present in TX table and successful response is sent to edg-perf01 {code} 13:55:41,177 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (INT-428,edg-perf03-30958) Attempting to execute command: CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:local, cacheName='testCache', topologyId=228} [sender=edg-perf01-60360] 13:55:41,197 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-978) Calling perform() on CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:remote, cacheName='testCache', topologyId=228} 13:55:41,197 TRACE [org.infinispan.commands.tx.AbstractTransactionBoundaryCommand] (remote-thread-978) Did not find a RemoteTransaction for GlobalTransaction:<edg-perf01-60360>:181323:remote 13:55:41,215 TRACE [org.jgroups.blocks.RequestCorrelator] (remote-thread-978) sending rsp for 977458 to edg-perf01-60360 13:55:41,215 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-978) About to send back response SuccessfulResponse{responseValue=null} for command CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:remote, cacheName='testCache', topologyId=228} {code} 6. edg-perf01 receives successful responses for commit command from both edg-perf02 & edg-perf03 {code} 13:55:41,230 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-1) Response(s) to CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:local, cacheName='testCache', topologyId=228} is {edg-perf03-30958=SuccessfulResponse{responseValue=null} , edg-perf02-25521=SuccessfulResponse{responseValue=null} } {code} 7. Transaction is commited again in new topology, further not affecting K. {code} 13:55:41,252 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-1) edg-perf01-60360 invoking CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:local, cacheName='testCache', topologyId=233} to recipient list [edg-perf02-25521, edg-perf03-30958, edg-perf04-55740] with options RpcOptions{timeout=60000, unit=MILLISECONDS, fifoOrder=false, totalOrder=false, responseFilter=null, responseMode=SYNCHRONOUS, skipReplicationQueue=false} ... 13:55:54,118 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (DefaultStressor-1) Response(s) to CommitCommand {gtx=GlobalTransaction:<edg-perf01-60360>:181323:local, cacheName='testCache', topologyId=233} is {edg-perf03-30958=SuccessfulResponse{responseValue=null} , edg-perf04-55740=SuccessfulResponse{responseValue=null} , edg-perf02-25521=SuccessfulResponse{responseValue=null} } {code} 8. Later, when get(K) is invoked on edg-perf01, both 02 & 03 send their response. However, each of them possesses a different version of the entry, leading to unexpected return value when message from 03 is received first. -- This message was sent by Atlassian JIRA (v6.3.11#6341)