[infinispan-issues] [JBoss JIRA] (ISPN-4310) StateResponse chunk with lastChunk=true from cancelled ST stops receiving data in next ST
[
https://issues.jboss.org/browse/ISPN-4310?page=com.atlassian.jira.plugin....
]
Dan Berindei updated ISPN-4310:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2584
Keep track of the topology id when state transfer was initiated, and reject state response
commands with older topology id.
We can't use the exact topology id, because a node (other than the coordinator)
leaving could increment the topology id without restarting the state transfer.
Note that the state transfer topology id is not necessarily the same as the topology id of
the {{REBALANCE_START}} command. If a node leaves, causing a
{{CH_UPDATE(rebalanceTopologyId + 1)}} command to be sent, other nodes may receive the
{{CH_UPDATE}} first and start the state transfer with the higher topology id.
StateResponse chunk with lastChunk=true from cancelled ST stops
receiving data in next ST
-----------------------------------------------------------------------------------------
Key: ISPN-4310
URL: https://issues.jboss.org/browse/ISPN-4310
Project: Infinispan
Issue Type: Bug
Components: State Transfer
Affects Versions: 6.0.2.Final, 7.0.0.Alpha4
Reporter: Radim Vansa
Assignee: Dan Berindei
Priority: Critical
1. A requests segment from B (there are multiple chunks)
2. B sends all chunks, but before A receives them, new topology arrives and A cancels the
ST.
3. Another topology comes and A requests this segment again
4. A receives the old StateResponseCommand with lastChunk=true and thinks that it got all
segments, therefore, it discards further chunks.
Result is inconsistent cluster, and after further rebalances completely lost data.
This ought to be rare, but was repeatedly observed when gracefully stopping coordinator
on a 32-node cluster full of data.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)