[teiid-issues] [JBoss JIRA] (TEIID-2460) Weird behavior when Max buffer space restriction is hit

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781123#comment-12781123 ] 

Steven Hawkins commented on TEIID-2460:
---------------------------------------

> Did you mean to free up the disk space? Or is this just addressing clearing of the heap?

It allows for the heap to also be cleaned up related to that tuplebuffer.  The only way to get the exception that you have attached is if the memory buffer/disk cleanup has happened.

> With current fixes in the backported 7.7.x I do not see such a behavior. 

I do not agree, unless there was an issue with the backport.  More that likely this is just the difference between 8.3+ behavior in which we create far fewer tuplebuffers for sorting thus cleanup takes effectively longer than just clearing a single tuplebuffer at the disk layer - the owning query must still be resumed and attempt to use the invalid tuplebuffer.  You should be able to confirm that the fix works in 7.7 if the disk limit is not exceeded, the exception reporting is not excessive, if the affected queries are aborted, and the disk/memory space after the queries are terminated becomes available for use.

> It is true that the query now fails immediatelly with exception [1], but the disk space is still occupied (thus it is not possible to submit another query that needs buffer space). Is this the intended behavior?

Other space occupied by the owning query will remain occupied until the overall query fails (which for 7.7 may take some time).  Also the root cleanup is asynch to begin with (there are three memory regions - heap, memory buffer, and disk - the last two are asynch to the first so any error that is happening at the disk layer is not synchronized to query processing), so it's highly likely likely that when hitting the out of disk condition that other queries will fail soon after under load.

Again, this is not intended to be a graceful handling of the situation, just a mechanism to prevent runaway disk consumption.  

                
> Weird behavior when Max buffer space restriction is hit 
> --------------------------------------------------------
>
>                 Key: TEIID-2460
>                 URL: https://issues.jboss.org/browse/TEIID-2460
>             Project: Teiid
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: 7.7.6
>            Reporter: Filip Nguyen
>            Assignee: Steven Hawkins
>             Fix For: 8.4, 7.7.7
>
>         Attachments: teiid-jboss-beans.xml
>
>
> I was trying to restrict the disk space used by buffer manager (see steps to reproduce for my methodology). When the disk limit is hit, it really tries to stop the query, but doesn't succeed immediately. 
> There is a big amount of exceptions [1] for relatively long period of time (minutes for big files), until it fails eventually with [2]. The error [2] is also given back to the JDBC client. 
> Problem is that after the query fails in this fashion, the whole buffer disk space is still occupied and any new query, that needs even small (acceptable) buffer disk space, fails.
> Only way that I have found to make the buffer space usable again is to restart the server.
> [1]  Error transferring block to storage 149742
> java.io.IOException: Max buffer space of 31,457,280 bytes has been exceed.  The current operation will be aborted.
> [2]  org.teiid.jdbc.TeiidSQLException: Batch not found in storage 50937

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira