[infinispan-dev] Infinispan and change data capture
Emmanuel Bernard
emmanuel at hibernate.org
Fri Dec 16 10:30:18 EST 2016
> On 16 Dec 2016, at 16:25, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>
>
>> On 16 Dec 2016, at 13:38, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>
>> On 16/12/16 13:12, Emmanuel Bernard wrote:
>>>
>>>> On 16 Dec 2016, at 09:48, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>>>
>>>> On 16/12/16 09:34, Emmanuel Bernard wrote:
>>>>>> Yes, the above design is what sprung to mind initially. Not sure about
>>>>>> the need of keeping the log in memory, as we would probably need some
>>>>>> form of persistent log for cache shutdown. Since this looks a lot like
>>>>>> the append-log of the Artemis journal, maybe we could use that.
>>>>>
>>>>> Well, when the cache is shut down, don’t we have time to empty the in-memory log?
>>>>
>>>> Cache shutdown should not be deferred because there is a backlog of
>>>> events that haven't been forwarded to Debezium, so we would want to pick
>>>> up from where we were when we restart the cache.
>>>
>>> But you’re willing to wait for the Artemis journal finish writing? I don’t quite see the difference.
>>
>> I'm thinking about the case where Debezium is temporarily not able to
>> collect the changes.
>
> I’m thinking about the case where Artemis is not able to collect the changes. How is that different? :)
So to answer my own questions.
The Emmanuel Dec 9th proposal handles I think the case of topology changes and nodes going down.
For a cluster-wide shutdown with no time to flush queues, I think both Artemis and the local debezium talking to remote Kafka will be in trouble.
The main difference is I imagine that your Artemis log will be local to the node being shut down. But that would mean a complex system to restart with the unflushed queue from all node that were shut down. Are we there yet?
More information about the infinispan-dev
mailing list