[teiid-issues] [JBoss JIRA] (TEIID-1750) Memory handling improvements

[Steven Hawkins (Commented) (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633191#comment-12633191 ] 

Steven Hawkins commented on TEIID-1750:
---------------------------------------

Reducing the memory overhead of ManagedBatches, using variable sized batches, switching to a concurrent LRFU, etc.  In initial testing so little have changed in the single threaded all memory case (depending on how trivial the data set is there can be a small degredation to a small improvement).  Minor additional overhead is to be expected due to maintaining the cache ordering.

In multi-threaded testing even with small amounts of data (2 threads creating/manipulating 2 256k row tables with 128byte rows) both the all memory and a constrained buffer memory test show marked improvement of 30% or more.
                
> Memory handling improvements
> ----------------------------
>
>                 Key: TEIID-1750
>                 URL: https://issues.jboss.org/browse/TEIID-1750
>             Project: Teiid
>          Issue Type: Quality Risk
>          Components: Query Engine
>    Affects Versions: 7.0
>            Reporter: Steven Hawkins
>            Assignee: Steven Hawkins
>             Fix For: 7.6
>
>
> The buffermanager will create softreferences for batches that prefer memory, but does not track the memory as held once the value has been passivated.
> Also small lobs, such as xml values created via sqlxml, should be inlined into results for greater performance - and so that they can be paged out by the buffermanager rather than being held by a lobmanager.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira