[teiid-issues] [JBoss JIRA] (TEIID-2584) Add management features to materialization

Thu Aug 22 09:49:26 EDT 2013

    [ https://issues.jboss.org/browse/TEIID-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798498#comment-12798498 ] 

Steven Hawkins commented on TEIID-2584:
---------------------------------------

> I really do not need those properties for this feature. 

If they're not needed, I would vote for keeping them out for now.

> From the privacy aspect, should we better off tie into some role to access any system tables in general?

SYS is considered public regardless of whether roles are enabled.  SYSADMIN is not.  So any system information that may have a security concern should go in SYSADMIN instead - for example the procedure to grab vdb contents, which can include the raw xmi files is in SYSADMIN.

> It is returning the number of rows loaded in that "load" call, not the status of the call.

I just mean that you can easily be more informational so that a call to load can give someone the status (loading e.g -1, loaded e.g. -2) rather than just returning 0.  And 0 is a valid number of rows loaded for an empty materialized view, but you wouldn't know what the call actually did.

> I made the status table "valid" column as not null, since it is doing null check it will never get into "insert" if there is a entry already.

Yes but simultaneous load calls will both attempt to do an insert.

> I did not choose VDB startup time to avoid adding another procedure

Yes I suppose that just adds another complication.  And in theory the management of the materialized tables can be quite complicated.  We have logic in the internal materialization stuff about sharing imported matviews, something like that may be needed here as well (although that complicates the notion of what vdb/version a materialization table is from).  Also people may want materialization tables to be shared among vdb versions and selectively destroyed with a new deployment.

> Thought about it, but was not sure if the current model of vdb.xml is good vehicle for it. Also, we need to worry about distinguishing between vdb reloads, and first time loads to expose as general purpose feature. Could be worked as separate issue.

It seems odd though to have a hook that is on a public api, but not expose it.  If it's not intended for general use or has issues, then we need to document what those are.

> How about this proposal? Add two additional metadata properties on View

I'm not sure that will be sufficient.  What would a partial load script do?

See http://wiki.postgresql.org/wiki/Materialized_Views especially the pdf link on http://www.pgcon.org/2008/schedule/events/69.en.html covering the use of a dirty column flag

We also do capture the update events going through Teiid, but not to a row level (since we don't try to determine the full effect of the source update).  That can be done through a source for each row trigger though if we want to go that route (at the cost of performance).

Another possibility is to expand the semantics of the nocache hint (or introduce something new) such that it instructs the engine to perform a refresh of just a range of the materialization table - rather than the simplistic approach of the internal logic that just updates 1 row at a time.

> Add management features to materialization
> ------------------------------------------
>
>                 Key: TEIID-2584
>                 URL: https://issues.jboss.org/browse/TEIID-2584
>             Project: Teiid
>          Issue Type: Feature Request
>            Reporter: Ramesh Reddy
>            Assignee: Ramesh Reddy
>              Labels: Beta3
>             Fix For: 8.5
>
>
> Currently Teiid supports internal and external materialization features. The internal is managed completely by the Teiid query engine along with the infinispan cache. 
> External materialization is completely unmanaged and left out to the user to manage it externally. This goals for this feature are unify the materialization logic for internal and external, such that both are managed similarly irrespective of the type of materialization chosen. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira