[teiid-issues] [JBoss JIRA] Commented: (TEIID-168) Remove resultset caching

[Greg Haber (JIRA)]

    [ https://jira.jboss.org/jira/browse/TEIID-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12453775#action_12453775 ] 

Greg Haber commented on TEIID-168:
----------------------------------

I'm actually not particularly interested in end user caching - I was just referring back to the example in Ramesh's comment on Feb 19.

Let me clarify the use case that I'm really interested in (or, more accurately, that a current customer of the MetaMatrix product is interested in solving down the road with Teiid):

-Most of the read requests at this customer are for a relatively small (several GB) set of reference data, with the data source guaranteed not to change during the business day (all updated happen during a small, well-defined overnight period).

-Many different end users at the customer access this reference data.

-The current data source systems are overworked so they want to avoid sending more requests back to these sources.

-They are not interested in setting up additional replication of the data into more relational databases - that has been their current approach to incrementally add more capacity, but every additional replication increases complexity of data distribution, and they end up with the relational database as a bottleneck.

So this is a case where server side intermediate result set caching with heavy use of in-memory caching mechanisms is a great fit.  They can warm the cache at the beginning of each day by issuing a set of queries for the data to be cached, and set cache expiration so that the results will be stored by cache for the duration of the business day.

And this is certainly doable without a distributed cache - they would just need to warm each JVM individually, instead of doing it on the Teeid cluster as a whole and relying on the distributed cache mechanism to get results to each JVM.  But that adds some complexity to the management of the caching jobs (especially as they grow to have potentially several dozen JVMs in their installation) - so long term, distributed cache functionality would be very useful.

In other words, it isn't the performance of the cache population that is the question, it is the management of the population activity with a large number of JVMs.





> Remove resultset caching
> ------------------------
>
>                 Key: TEIID-168
>                 URL: https://jira.jboss.org/jira/browse/TEIID-168
>             Project: Teiid
>          Issue Type: Feature Request
>          Components: JDBC Connector, Query Engine
>    Affects Versions: 6.x
>            Reporter: Steven Hawkins
>            Assignee: Steven Hawkins
>             Fix For: 6.1.0
>
>
> Resultset caching should be replaced with relation and procedure caching.  This would initially be backed by the buffermanager, based upon hints/costing, and be utilized by the planner rather than the resolver.  It would be good if we also detected the level of determinism.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira