[infinispan-dev] Infinispan and change data capture

[Emmanuel Bernard]

> On 15 Dec 2016, at 11:18, Gustavo Fernandes <gustavo at infinispan.org> wrote:
> 
> On Thu, Dec 15, 2016 at 9:54 AM, Emmanuel Bernard <emmanuel at hibernate.org <mailto:emmanuel at hibernate.org>> wrote:
> The goal is as followed: allow to collect all changes to push them to Debezium and thus Kafka.
> 
> This need does not require to remember all changes since the beginning of time in Infinispan. Just enough to:
> - let Kafka catchup assuming it is the bottleneck
> - let us not lose a change in Kafka when it happened in Infinispan (coordinator, owner, replicas dying)
> 
> The ability to read back history would then be handled by the Debezium / Kafka tail, not infinispan itself.
> 
> 
> Having an embedded Debezium connector pushing everything to Kafka sounds cool, but what impact would it bring to the other stream consumers:
> 
> * Remote listeners, which is supported in several clients apart from Java
> * Continuous Queries (the same)
> * Spark Stream
> * Other eventual 3rd party stream processors: Apache Flick, Storm, etc.
> 
>  

Impact as in perf impact? Potential redesign impact? Or are you thinking of another question?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20161215/01243028/attachment.html