<div dir="auto"><div><br><div class="gmail_extra"><br><div class="gmail_quote">On 16 Dec 2016 12:39, "Tristan Tarrant" <<a href="mailto:ttarrant@redhat.com">ttarrant@redhat.com</a>> wrote:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="quoted-text">On 16/12/16 13:12, Emmanuel Bernard wrote:<br>
><br>
>> On 16 Dec 2016, at 09:48, Tristan Tarrant <<a href="mailto:ttarrant@redhat.com">ttarrant@redhat.com</a>> wrote:<br>
>><br>
>> On 16/12/16 09:34, Emmanuel Bernard wrote:<br>
>>>> Yes, the above design is what sprung to mind initially. Not sure about<br>
>>>> the need of keeping the log in memory, as we would probably need some<br>
>>>> form of persistent log for cache shutdown. Since this looks a lot like<br>
>>>> the append-log of the Artemis journal, maybe we could use that.<br>
>>><br>
>>> Well, when the cache is shut down, don’t we have time to empty the in-memory log?<br>
>><br>
>> Cache shutdown should not be deferred because there is a backlog of<br>
>> events that haven't been forwarded to Debezium, so we would want to pick<br>
>> up from where we were when we restart the cache.<br>
><br>
> But you’re willing to wait for the Artemis journal finish writing? I don’t quite see the difference.<br>
<br>
</div>I'm thinking about the case where Debezium is temporarily not able to<br>
collect the changes.<br></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">+1 That's the crucial concern. </div><div dir="auto"><br></div><div dir="auto">We can have Infinispan attempt to transmit all updates to debezium on a best effort base, but we can't guarantee to send them all. We can resume the state replication stream as Randall suggested, providing in that case a squashed view which might work great to replicate the same state. </div><div dir="auto">Analysis of the stream of updates though shall not be able to rely on seeing *all* events. </div><div dir="auto">As Emmanuel seemed to agree previously, for that use case one would need a different product like Kafka. </div><div dir="auto"><br></div><div dir="auto">What I'm getting at is that if we agree that this granularity of events needs *just* to replicate state, then we can take advantage of that detail in various areas of the implementation, providing significant performance optimisation opportunities. </div><div dir="auto"><br></div><div dir="auto">Sanne </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="quoted-text"><br>
Tristan<br>
--<br>
Tristan Tarrant<br>
Infinispan Lead<br>
JBoss, a division of Red Hat<br>
</div><div class="elided-text">______________________________<wbr>_________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/infinispan-<wbr>dev</a></div></blockquote></div><br></div></div></div>