On 09/12/16 18:25, Emmanuel Bernard wrote:
The total order would not be global but per key.
Each node has a Debezium connector instance embedded that listens to the
operations happening (primary and replicas alike).
All of this process is happening async compared to the operation.
Per key, a log of operations is kept in memory (it contains the key, the
operation, the operation unique id and a ack status.
If on the key owner, the operation is written by the Debezium connector
to Kafka when it has been acked (whatever that means is where I'm less
knowledgable - too many bi-cache, tri-cache and quadri latency mixed in
my brain).
On a replica, the kafka partition is read regularly to clear the
in-memory log from operations stored in Kafka
If the replica becomes the owner, it reads the kafka partition to see
Yes, the above design is what sprung to mind initially. Not sure about
the need of keeping the log in memory, as we would probably need some
form of persistent log for cache shutdown. Since this looks a lot like
the append-log of the Artemis journal, maybe we could use that.
One thing I missed, is whether Debezium is interested in transaction
boundaries.
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat