[teiid-issues] [JBoss JIRA] (TEIIDSB-147) Caching and materialization changes for JDG / OpenShift

[Steven Hawkins (Jira)]

    [ https://issues.jboss.org/browse/TEIIDSB-147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821103#comment-13821103 ] 

Steven Hawkins commented on TEIIDSB-147:
----------------------------------------

> Whether JDG is multi-tenant or not IMO it should not matter for us, DV cache instance is DV only.

Should have been more specific - I was asking about the relationship between the VDBs and JDG instances.  

Based upon the response from the prior the expectation is approximately that admin is responsible for creating a JDG instance.  From there we're assuming a cache instance per VDB.  Then how JDG manages the cache instances is up to JDG.

> we can move the materialization management aspect to the Operator to manage, so failure can be detected

Can you elaborate on that?  Are you saying that the operator will take over as the initiator of materialization actions, time-based refreshes, etc.  Or are you saying that the operator will somehow provide the pods with a list of peer pods and their status? 

> unless we want to use the buffer manager with local disk, i would guess replication is not there.

I should say that result set caching is more of a multi-pod problem.   In a single pod scenario the worst that will happen right now is that the cache won't survive a pod restart - but that's no different than having a single JDV instance today.  

Replication isn't so much the concern as distribution - things need to be reimplemented so that the batches are stored in JDG.  As it stands the results would not be shared across pods.  Whether we also create local entries in the buffermanager (or a local cache) is optional.

> Caching and materialization changes for JDG / OpenShift
> -------------------------------------------------------
>
>                 Key: TEIIDSB-147
>                 URL: https://issues.jboss.org/browse/TEIIDSB-147
>             Project: Teiid Spring Boot
>          Issue Type: Enhancement
>          Components: core, OpenShift
>            Reporter: Steven Hawkins
>            Assignee: Ramesh Reddy
>            Priority: Major
>             Fix For: 1.3.0
>
>
> Started this as a higher level issue in Teiid Spring Boot rather than core Teiid since the work won't align well to Teiid 13.1, but some things may need addressed there as well.
> Things break down roughly into:
> * Single-pod support
> ** Define the expectations for how we'll be configured to hit a JDG instance - will it be a user exercise to create if it doesn't exist, will we need to create, will the be a crd strategy for creation, and will it be multi-tenet?  There is also the issue / assumption of cache sharing based upon vdb name - is that sufficient for now?
> ** Result set caching needs to directly write batches / results to JDG rather than relying upon on demand replication.
> ** validate external materialization to JDG likely using a transactional upsert strategy to avoid issues like TEIID-5436
> * Multi-pod support
> ** everything from single-pod support
> ** requires replacing the logic that detect failed materialization loads
> ** internal materialization replacement needs to based upon something more like external materialization, but that can be seen as a next step that is turnkey internal materialization to JDG (little to no user setup required)



--
This message was sent by Atlassian Jira
(v7.13.8#713008)