[teiid-issues] [JBoss JIRA] (TEIID-2460) Weird behavior when Max buffer space restriction is hit

Filip Nguyen (JIRA) jira-events at lists.jboss.org
Wed Jun 12 04:22:55 EDT 2013 

    [ https://issues.jboss.org/browse/TEIID-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781006#comment-12781006 ] 

Filip Nguyen commented on TEIID-2460:
-------------------------------------

What is meant by:

 #2) added a callback into the buffer manager to clear out all memory related to the failing tuple batch and ensured if additional batches are added that the process will fail (not just if a batch is retrieved).

Did you mean to free up the disk space? Or is this just addressing clearing of the heap? With current fixes in the backported 7.7.x I do not see such a behavior. It is true that the query now fails immediatelly with exception [1], but the disk space is still occupied (thus it is not possible to submit another query that needs buffer space). Is this the intended behavior?

[1] 

{code}
org.teiid.jdbc.TeiidSQLException: Cannot add batch to invalidated cache group "{1}".  Check prior logs to see if there was an error persisting a batch.
	at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:113)
	at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:70)
	at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:643)
	at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
	at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:547)
	at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:130)
	at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:37)
	at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
	at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:222)
	at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:257)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:101)
	at com.sun.proxy.$Proxy1.read(Unknown Source)
	at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:356)
	at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:556)
	at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:334)
	at JDBCClient.execute(JDBCClient.java:77)
	at JDBCClient.main(JDBCClient.java:45)
Caused by: org.teiid.core.TeiidComponentException: Cannot add batch to invalidated cache group "{1}".  Check prior logs to see if there was an error persisting a batch.
	at org.teiid.common.buffer.impl.BufferManagerImpl$BatchManagerImpl.createManagedBatch(BufferManagerImpl.java:211)
	at org.teiid.common.buffer.TupleBuffer.saveBatch(TupleBuffer.java:254)
	at org.teiid.common.buffer.TupleBuffer.addTuple(TupleBuffer.java:191)
	at org.teiid.common.buffer.TupleBuffer.addTupleBatch(TupleBuffer.java:204)
	at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:73)
	at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
	at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:85)
	at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91)
	at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120)
	at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:210)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70)
	at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
	at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:85)
	at org.teiid.common.buffer.AbstractTupleSource.nextTuple(AbstractTupleSource.java:48)
	at org.teiid.query.processor.relational.SortUtility.initialSort(SortUtility.java:248)
	at org.teiid.query.processor.relational.SortUtility.sort(SortUtility.java:185)
	at org.teiid.query.processor.relational.SortNode.sortPhase(SortNode.java:99)
	at org.teiid.query.processor.relational.SortNode.getFinalBuffer(SortNode.java:180)
	at org.teiid.query.processor.relational.RelationalPlan.getFinalBuffer(RelationalPlan.java:280)
	at org.teiid.query.processor.QueryProcessor.getFinalBuffer(QueryProcessor.java:239)
	at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:137)
	at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:377)
	at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:290)
	at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)
	at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:218)
	at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:244)
	at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)
	at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

{code}

                
> Weird behavior when Max buffer space restriction is hit 
> --------------------------------------------------------
>
>                 Key: TEIID-2460
>                 URL: https://issues.jboss.org/browse/TEIID-2460
>             Project: Teiid
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: 7.7.6
>            Reporter: Filip Nguyen
>            Assignee: Steven Hawkins
>             Fix For: 8.4, 7.7.7
>
>         Attachments: teiid-jboss-beans.xml
>
>
> I was trying to restrict the disk space used by buffer manager (see steps to reproduce for my methodology). When the disk limit is hit, it really tries to stop the query, but doesn't succeed immediately. 
> There is a big amount of exceptions [1] for relatively long period of time (minutes for big files), until it fails eventually with [2]. The error [2] is also given back to the JDBC client. 
> Problem is that after the query fails in this fashion, the whole buffer disk space is still occupied and any new query, that needs even small (acceptable) buffer disk space, fails.
> Only way that I have found to make the buffer space usable again is to restart the server.
> [1]  Error transferring block to storage 149742
> java.io.IOException: Max buffer space of 31,457,280 bytes has been exceed.  The current operation will be aborted.
> [2]  org.teiid.jdbc.TeiidSQLException: Batch not found in storage 50937

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

More information about the teiid-issues mailing list