]
Dan Berindei commented on ISPN-8232:
------------------------------------
True, the stable topology doesn't fix the problem by itself. But having a stable
topology does allow us to do some extra processing before updating the stable topology in
the majority partition, like I suggested in ISPN-3421/ISPN-5046.
OTOH I don't see how an "expected members" list different from the list of
members in the stable topology could be maintained without the administrator having to
manually intervene every time a node crashes.
Transaction inconsistency during network partitions
---------------------------------------------------
Key: ISPN-8232
URL:
https://issues.jboss.org/browse/ISPN-8232
Project: Infinispan
Issue Type: Bug
Components: Transactions
Affects Versions: 9.1.0.Final
Reporter: Pedro Ruivo
Assignee: Pedro Ruivo
Priority: Critical
In scenario where the originator stays in minor partition (in our test suite, the
originator isolated tests), it is possible to a transaction to be committed and rolled
back in the majority partition.
In {{Pessimitic Locking}}, the transaction is committed in one-phase using the
{{PrepareCommand}}. If the partition happens when the originator sends the
{{PrepareCommand}}, the nodes in the majority partition may or may not receive it. We can
have the case where some nodes receive the {{PrepareCommand}} and applied and other
don't receive it.
When the topology is updated in the majority partition, the {{TransactionTable}}
rollbacks all transaction in which the originator isn't present. So, in the nodes
where the {{PrepareCommand}} isn't received, the transaction is rolled back.
The originator in the minory partition detects the partition and marks the transaction
partially completed. When the merge occurs, it tries to commit the transaction again. In
the nodes where the transaction is rolled back, the transaction is marked as completed and
when the {{PrepareCommand}} is received, it throws an {{IllegalStateException}}
({{TransactionTable:386, getOrCreateRemoteTransaction()}}). In this case, the transaction
isn't removed from the {{PartitionHandlingManager}} and our test suite fails with
{{"there are pending tx".}}
Other theoretically scenario is the {{PrepareCommand}} to be executed when no locks are
acquired.
The same issue can happen with {{Optimistic Locking}} for the {{CommitCommand}}.
The problem is the transaction table can't identify is the node left gracefully or
not. A solution would be to have an {{"expected members"}} list, ideally
separated from the {{CacheTopology}} to avoid sending it every time. Also, it would need
some sysadmin tools for the case where the node crashes and it won't be back online
for a while (or for some reason, it doesn't need to be back online).
A sysadmin could remove the node from this list ({{CacheTopology}} is updated and there
is no need to increase it) and decide what to do with the pending transactions (or an
automatic mechanism to auto-commit/rollback the transaction).