[JBoss JIRA] (TEIIDSB-51) Define expectations for runtime metadata
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-51?page=com.atlassian.jira.plugi... ]
Ramesh Reddy updated TEIIDSB-51:
--------------------------------
Fix Version/s: 1.4.0
(was: 1.3.1)
> Define expectations for runtime metadata
> ----------------------------------------
>
> Key: TEIIDSB-51
> URL: https://issues.redhat.com/browse/TEIIDSB-51
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Components: documentation
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.4.0
>
>
> At a high level there are potentially three modes for our ddl:
> 1. Interactive - there was some work toward this, but it needed to only be a "developer mode" as it would reload the vdb on each metadata change. There is a lot about the transactionality of DDL updates that our current simplistic metadata model does not support, which makes this hard to fully implement. We haven't talked about this much in relation to Teiid Spring Boot, but ideally there should be a way for developers to test incremental changes without full rebuilds.
> 2. Dynamic - the term is a bit dated, but this refers to any vdb that performs import from a metadata repository that has runtime dependencies. By design this allows for a simple vdb definition and options for caching or re-importing on reload. In a container environment metadata caching no longer makes sense unless we use a volume. Depending upon your viewpoint allowing for runtime import of metadata is potentially a point of mutability that is also not needed in a container environment.
> 3. Static - much like our old .index designer vdbs there are benefits to having the full metadata already available to the vdb/image. There is a lot of work that we do lazily at runtime that can be moved to an earlier phase - primarily generating the rest and odata layers, but it would also include the pg system table materializations. For Teiid Spring Boot or usage in containers in general what this would look like is a build phase for the vdb such that it could obtain appropriate metadata and initialize as much as possible statically (eventually for quarkus vm snapshotting). The only metadata that would not be fresh potentially would be the costing metadata - however that is problematic as is and will eventually need a progressive optimization strategy employing a persistent store.
> As of now we are currently focused just on #2. As we look toward operators, we should start thinking about #3.
> #1 would come into play if/when we start looking at local development options using vs code.
> We can treat this issue as an epic and a place to centralize discussion.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-147) Caching and materialization changes for JDG / OpenShift
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-147?page=com.atlassian.jira.plug... ]
Ramesh Reddy updated TEIIDSB-147:
---------------------------------
Fix Version/s: 1.4.0
(was: 1.3.1)
> Caching and materialization changes for JDG / OpenShift
> -------------------------------------------------------
>
> Key: TEIIDSB-147
> URL: https://issues.redhat.com/browse/TEIIDSB-147
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: core, OpenShift
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 1.4.0
>
>
> Started this as a higher level issue in Teiid Spring Boot rather than core Teiid since the work won't align well to Teiid 13.1, but some things may need addressed there as well.
> Things break down roughly into:
> * Single-pod support
> ** Define the expectations for how we'll be configured to hit a JDG instance - will it be a user exercise to create if it doesn't exist, will we need to create, will the be a crd strategy for creation, and will it be multi-tenet? There is also the issue / assumption of cache sharing based upon vdb name - is that sufficient for now?
> ** validate external materialization to JDG likely using a transactional upsert strategy to avoid issues like TEIID-5436
> * Multi-pod support
> ** everything from single-pod support
> ** Result set caching needs to directly write batches / results to JDG rather than relying upon on demand replication.
> ** requires replacing the logic that detect failed materialization loads
> ** internal materialization replacement needs to based upon something more like external materialization, but that can be seen as a next step that is turnkey internal materialization to JDG (little to no user setup required)
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months