[teiid-issues] [JBoss JIRA] (TEIID-4526) Integrate with Debezium for CDC for maintaining materialized views

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13318403#comment-13318403 ] 

Steven Hawkins commented on TEIID-4526:
---------------------------------------

In developing a plan for CDC integration near term work could include:

- further enhance/refine the built in materialization management (both internal and external) to support alternative invalidation and refresh strategies.  Full snapshot refresh is supported by both internal/external, and a predicate based invalidation is supported for external.  The latter potentially allows for more control, but does not re-enforce a particular strategy, must be called manually, and relies upon a built-in delete, then insert - which is problematic when a source is not transactional and should be changed to a merge/delete approach.  Other approaches such as lazy invalidation down to a row level require the materialization table to some natural or additional column that allows for row to be identified as stale.
- However providing out of the box built-in eager or lazy invalidation to a row level implies that routines for CDC events that goes well beyond our current logic around inherently updatable views.  I don't think we should assume that it would be developed for Teiid 9.2, which should imply that we do want high-level plugability of event handling.  This could even be handled at the language level with new trigger semantics for source tables: create trigger on tbl on cdc (insert|update...) - which could potentially even specify the topic mapping other details.  There is complexity for multi-source models here though.
- Provide debezium based out of the box support for result set cache invalidation built upon the current table level update event logic.
- consider the benefits of consolidating status table management as suggested by [~kylin]

> Integrate with Debezium for CDC for maintaining materialized views
> ------------------------------------------------------------------
>
>                 Key: TEIID-4526
>                 URL: https://issues.jboss.org/browse/TEIID-4526
>             Project: Teiid
>          Issue Type: Feature Request
>          Components: Server
>    Affects Versions: 9.2
>            Reporter: Van Halbert
>            Assignee: Steven Hawkins
>            Priority: Critical
>
> Integrate with Debezium so that Teiid will be able to consume and react to the row-level change events and do something interesting with them, such as update the materialized view(s).



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)