[teiid-issues] [JBoss JIRA] (TEIID-2460) Weird behavior when Max buffer space restriction is hit

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765247#comment-12765247 ] 

Steven Hawkins commented on TEIID-2460:
---------------------------------------

> There is a big amount of exceptions [1] for relatively long period of time (minutes for big files), until it fails eventually with [2]. The error [2] is also given back to the JDBC client.

The plan or asynch thread processing the eviction to disk will likely not be the owner of the given batch, thus we log that the problem occurred and don't take immediate action - we are also ideally in a scenario where the plan will not need that particular batch or it's recoverable via a weak reference.  Only when the plan requests a batch that is no longer available do we throw back the exception and cleanup.  The delay between when the batch is dropped and when the plan needs it should be no different than in normal processing (and appears to work that way for me on 8.4).

> Problem is that after the query fails in this fashion, the whole buffer disk space is still occupied and any new query, that needs even small (acceptable) buffer disk space, fails.

I don't see this on 8.4 either.  However at worst this may be up to the platform related.  If we don't have an immediate removal of defunct TupleBuffers, it is up to garage collection to trigger the cleanup.  So you could for example be hitting https://issues.jboss.org/browse/TEIID-2430 but even then you should see the space being reclaimed in a relatively short time.

Can you retest on 8.4?
                
> Weird behavior when Max buffer space restriction is hit 
> --------------------------------------------------------
>
>                 Key: TEIID-2460
>                 URL: https://issues.jboss.org/browse/TEIID-2460
>             Project: Teiid
>          Issue Type: Bug
>    Affects Versions: 7.7.6
>            Reporter: Filip Nguyen
>            Assignee: Steven Hawkins
>
> I was trying to restrict the disk space used by buffer manager (see steps to reproduce for my methodology). When the disk limit is hit, it really tries to stop the query, but doesn't succeed immediately. 
> There is a big amount of exceptions [1] for relatively long period of time (minutes for big files), until it fails eventually with [2]. The error [2] is also given back to the JDBC client. 
> Problem is that after the query fails in this fashion, the whole buffer disk space is still occupied and any new query, that needs even small (acceptable) buffer disk space, fails.
> Only way that I have found to make the buffer space usable again is to restart the server.
> [1]  Error transferring block to storage 149742
> java.io.IOException: Max buffer space of 31,457,280 bytes has been exceed.  The current operation will be aborted.
> [2]  org.teiid.jdbc.TeiidSQLException: Batch not found in storage 50937

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira