[
https://issues.jboss.org/browse/TEIID-4526?page=com.atlassian.jira.plugin...
]
Steven Hawkins commented on TEIID-4526:
---------------------------------------
the above approach of using updatematViewRow is not correct because
its updating the actual row in the mat view table without applying the actual View
transformation logic
The considerations here are that the updateMatViewRow is only for internal
materialization, and yes it's directly tied to modifying the materialization table.
To create an appropriate change value, you would need to do a select with no cache from
view and pull appropriately updated rows, then update each one. This method largely
exists because there was no other way to modify the internal materialization target other
than to completely refresh and the change is easy to broadcast.
The only way to ensure the correct result is to only trigger the
update when the "key" can be identified in a change.
Yes, some "trigger" has to understand a base tables relationship to all
consuming entities and propagate them accordingly. As mentioned above this can be done
with a full row fresh, or what I've talked about is updating of materialization logic
to support lazier row refreshing using an invalidation flag - but it still requires
determining what row(s) are affected.
Am I over thinking this? Is there something simpler?
Unfortunately it's much more complicated. Our problems with CDC starts with the
metadata. Since we allow for hand editing and only capture a simple quoted name in
source, we first have a matching problem from any event source to our source metadata.
Then inferring values directly from the change event can be an issue as the CDC layer is
broadcasting raw values, not what you would fetch over jdbc (there can be type issues, or
even the consideration of masking and other affects). From there yes we have to
sufficiently understand the view structure to make an appropriate update. That will not
always be a simple changes as aggregates may need recomputed, multiple rows removed, etc.
What we talked about at the face-to-face was at least keeping some notion of how
"dirty" a particular materialized view was (number of updates that affect the
view vs. number of view rows) and trigger a full refresh above a given threshold. We also
need to determine if there are differing expectations for internal vs. external
materialization.
Integrate with Debezium for CDC for maintaining materialized views
------------------------------------------------------------------
Key: TEIID-4526
URL:
https://issues.jboss.org/browse/TEIID-4526
Project: Teiid
Issue Type: Feature Request
Components: Server
Affects Versions: 9.2
Reporter: Van Halbert
Assignee: Steven Hawkins
Priority: Critical
Integrate with Debezium so that Teiid will be able to consume and react to the row-level
change events and do something interesting with them, such as update the materialized
view(s).
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)