[teiid-issues] [JBoss JIRA] (TEIID-2169) Buffer Manger's memory should be backed by Infinispan Cache

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742175#comment-12742175 ] 

Steven Hawkins commented on TEIID-2169:
---------------------------------------

Based upon https://community.jboss.org/thread/178102 and several INSP issues about memory pressure eviction, this is not something that we can currently do or even address in the near term.  The eviction strategy is not customizable and the eviction queue is not accessible (the Eviction is not accessible from a Segment and the queue is not accessible from the Eviction).

The core issue here is whether the LFRU imposes too much overhead vs. LIRS.  We should do some performance tests on hit ratios and overhead to see if we need to push this feature into INSP.

Ideally the entire strategy could be implemented with INSP, however they have already rejected the use of NIO managed direct memory with the possible inclusion of JNI accessed current maps from intel c libraries.  There again it could be evaluated whether adapting Teiid's existing use of direct memory (which isn't quite complete as keys are still held on heap and would need to be adapted for chained eviction) could be beneficial.
                
> Buffer Manger's memory should be backed by Infinispan Cache
> -----------------------------------------------------------
>
>                 Key: TEIID-2169
>                 URL: https://issues.jboss.org/browse/TEIID-2169
>             Project: Teiid
>          Issue Type: Feature Request
>          Components: Query Engine, Server
>            Reporter: Ramesh Reddy
>            Assignee: Steven Hawkins
>
> Currently the buffer manager uses the memory from the local VM's heap for processing the intermediate query results, anything that overruns is saved on the disk. Using this model so far proven very efficient. 
> However, when it comes to using the data grid as unlimited temp backing store where the results can be shared with other nodes in the case of fail-over of the session, this in current model does not work. By using the infinispan's local cache in standalone and replicated cache in distributed node for the buffer manager these issues can resolved.
> Issues may be, currently Buffer Manager uses efficient btree based index store, also the uses file storage in block. To make it work, Teiid needs to have access to the Infinispan's eviction queue and may also needs to implement the cache store.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira