[teiid-issues] [JBoss JIRA] (TEIID-4628) Support incremental materialization refresh strategies

[Steven Hawkins (JIRA)]

     [ https://issues.jboss.org/browse/TEIID-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steven Hawkins updated TEIID-4628:
----------------------------------
    Component/s: Query Engine
                 Server


> Support incremental materialization refresh strategies
> ------------------------------------------------------
>
>                 Key: TEIID-4628
>                 URL: https://issues.jboss.org/browse/TEIID-4628
>             Project: Teiid
>          Issue Type: Feature Request
>          Components: Query Engine, Server
>            Reporter: Steven Hawkins
>            Assignee: Steven Hawkins
>
> Further enhance/refine the built in materialization management to support alternative invalidation and refresh strategies. 
> In terms of priority, this would target external materialization first.
> There must be an alternative to the implied default of the table rename strategy - which is problematic when a source is not transactional and should be changed to a merge/delete approach. 
> The general approaches are lazy (rows marked as dirty with refresh on query/schedule) and eager (which can even attempt to maintain close to transactional consistency).  Optionally incremental loading can be used as well, to only load the portions of the materialization that are requested.
> This requires more logic that we currently have:
> - mapping source level row events to row level effects in materialized views (at worst we can refresh the whole row)
> - additional status information possibly down to a column level, which much be added to the materialization table.
> - incremental loading requires tracking the effectively covered predicate ranges and even columns of what has been loaded
> It would probably be good to focus on a single scenario at first, such as:
> external materialization, with non-incremental load (to reuse the existing simplistic load, but not rename logic), and eager refresh - and only considering transitive row effects based upon key relationships.
> Then we can ensure that sufficient options are being added to account for additional scenarios.



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)