[JBoss JIRA] (TEIID-2460) Weird behavior when Max buffer space restriction is hit
by Filip Nguyen (JIRA)
[ https://issues.jboss.org/browse/TEIID-2460?page=com.atlassian.jira.plugin... ]
Filip Nguyen updated TEIID-2460:
--------------------------------
Steps to Reproduce:
Create a text data source.
Create a dynamic VDB with file translator to that data source.
create a big text file:
for i in \{1..1000};do echo "1" >> result0;done
for i in \{1..8};do cat * > result$\{i};done
cat * > textfile
rm result*
configure the buffer manager: /home/fnguyen/work/edsqa/bz907640/jboss-soa-p-5/jboss-as/server/production/deploy/teiid/teiid-jboss-beans.xml
run query via JDBC: ./run.sh localhost 31000 FileTest "SELECT tex.first from (call getTextFiles('textfile')) as file, TEXTTABLE(file COLUMNS first integer) as tex order by tex.first desc"
was:
Create a text data source.
Create a dynamic VDB with file translator to that data source.
create a big text file:
for i in {1..1000};do echo "1" >> result0;done
for i in {1..8};do cat * > result${i};done
cat * > textfile
rm result*
configure the buffer manager: /home/fnguyen/work/edsqa/bz907640/jboss-soa-p-5/jboss-as/server/production/deploy/teiid/teiid-jboss-beans.xml
run query via JDBC: ./run.sh localhost 31000 FileTest "SELECT tex.first from (call getTextFiles('textfile')) as file, TEXTTABLE(file COLUMNS first integer) as tex order by tex.first desc"
> Weird behavior when Max buffer space restriction is hit
> --------------------------------------------------------
>
> Key: TEIID-2460
> URL: https://issues.jboss.org/browse/TEIID-2460
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 7.7.6
> Reporter: Filip Nguyen
> Assignee: Steven Hawkins
>
> I was trying to restrict the disk space used by buffer manager (see steps to reproduce for my methodology). When the disk limit is hit, it really tries to stop the query, but doesn't succeed immediately.
> There is a big amount of exceptions [1] for relatively long period of time (minutes for big files), until it fails eventually with [2]. The error [2] is also given back to the JDBC client.
> Problem is that after the query fails in this fashion, the whole buffer disk space is still occupied and any new query, that needs even small (acceptable) buffer disk space, fails.
> Only way that I have found to make the buffer space usable again is to restart the server.
> [1] Error transferring block to storage 149742
> java.io.IOException: Max buffer space of 31,457,280 bytes has been exceed. The current operation will be aborted.
> [2] org.teiid.jdbc.TeiidSQLException: Batch not found in storage 50937
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2460) Weird behavior when Max buffer space restriction is hit
by Filip Nguyen (JIRA)
Filip Nguyen created TEIID-2460:
-----------------------------------
Summary: Weird behavior when Max buffer space restriction is hit
Key: TEIID-2460
URL: https://issues.jboss.org/browse/TEIID-2460
Project: Teiid
Issue Type: Bug
Affects Versions: 7.7.6
Reporter: Filip Nguyen
Assignee: Steven Hawkins
I was trying to restrict the disk space used by buffer manager (see steps to reproduce for my methodology). When the disk limit is hit, it really tries to stop the query, but doesn't succeed immediately.
There is a big amount of exceptions [1] for relatively long period of time (minutes for big files), until it fails eventually with [2]. The error [2] is also given back to the JDBC client.
Problem is that after the query fails in this fashion, the whole buffer disk space is still occupied and any new query, that needs even small (acceptable) buffer disk space, fails.
Only way that I have found to make the buffer space usable again is to restart the server.
[1] Error transferring block to storage 149742
java.io.IOException: Max buffer space of 31,457,280 bytes has been exceed. The current operation will be aborted.
[2] org.teiid.jdbc.TeiidSQLException: Batch not found in storage 50937
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2381) Expanded source hint support
by Mark Addleman (JIRA)
[ https://issues.jboss.org/browse/TEIID-2381?page=com.atlassian.jira.plugin... ]
Mark Addleman commented on TEIID-2381:
--------------------------------------
> It's two-fold, the primary concern is the placement on subqueries, union branches, with clause items, etc. The secondary is for sources that support more complicated positional hints, such as a column hint. The primary is fairly feasible. Arbitrary positions however are currently not.
For us, table hints that get collected by the optimizer and delivered to the appropriate translator is the use case we have right now. I can imagine use for positional hints but those are less compelling to me.
> If you want a hint to appear in a push-down subqueries, that would require additional translator support - e.g. a query and its subquery both have hints and are pushed together - "SELECT /*+ sh ... */ ... (SELECT /*+ sh ... */ ... ) FROM ..." such that the pushdown has both hints.
This is appealing but, as you say, the translator would have to be smart enough to retrieve the hints. The alternative, however, seems undesirable: the optimizer would have to understand the semantics of the hints and make decisions about which to push or not or how to combine. I think that would either imply some standardization of hints or some API to describe metadata about hints. Neither of those options seem like a useful road to go down.
> For example if you have two views each with their own source hint and you join them together. The optimizer removes the view layers and reorders the join. Where do the hints apply and why?
Although it's a little embarrassing to admit, I'm still not versed enough in SQL to think about this in the abstract. Can you provide a specific example?
> Expanded source hint support
> ----------------------------
>
> Key: TEIID-2381
> URL: https://issues.jboss.org/browse/TEIID-2381
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Attachments: HintsExecutionFactory.java, RandomExecutionFactory.java, RandomNumberStoredProcedureExecution.java, SysviewHintsExecutionTest.java
>
>
> We currently look at the source hint in only the root user query (not in subqueries nor the with clause) and only consider it in a very narrow set of circumstances when it's used in a view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2457) Assertion error with INTERSECT
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2457?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-2457:
----------------------------------
Summary: Assertion error with INTERSECT (was: Using jdbc-simple translator produces Assertion error)
Priority: Major (was: Minor)
> Assertion error with INTERSECT
> ------------------------------
>
> Key: TEIID-2457
> URL: https://issues.jboss.org/browse/TEIID-2457
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 7.7.6
> Reporter: Van Halbert
> Assignee: Steven Hawkins
>
> A sample query which produced the error is: SELECT IntKey FROM BQT1.SmallA INTERSECT SELECT IntNum FROM BQT1.SmallB ORDER BY INTKEY
> How to reproduce: Run the query using a client such as Squirrel multiple times. The results varies from returning an Assertion to returning actual results. In my test I ran the query five times and it returned actual results twice and the Assertion 3 times. The Assertion can be seen below.
> 2013-04-03 16:00:42,190 ERROR [org.teiid.PROCESSOR] (Worker4_QueryProcessorQueue146) Unexpected exception for request JyDs6MZ6d9zG.24
> java.lang.AssertionError: ASSERTION FAILED: expected reference to be not null
> at org.teiid.core.util.Assertion.failed(Assertion.java:73)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:100)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:92)
> at org.teiid.common.buffer.TupleBuffer.getBatch(TupleBuffer.java:217)
> at org.teiid.common.buffer.TupleBuffer$1.getBatch(TupleBuffer.java:331)
> at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:61)
> at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91)
> at org.teiid.query.processor.relational.MergeJoinStrategy.process(MergeJoinStrategy.java:163)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:212)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
> at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70)
> at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
> at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:85)
> at org.teiid.common.buffer.AbstractTupleSource.nextTuple(AbstractTupleSource.java:48)
> at org.teiid.query.processor.relational.SortUtility.initialSort(SortUtility.java:248)
> at org.teiid.query.processor.relational.SortUtility.sort(SortUtility.java:185)
> at org.teiid.query.processor.relational.SortNode.sortPhase(SortNode.java:99)
> at org.teiid.query.processor.relational.SortNode.nextBatchDirect(SortNode.java:88)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
> at org.teiid.query.processor.relational.LimitNode.nextBatchDirect(LimitNode.java:100)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:244)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2457) Assertion error with INTERSECT
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2457?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-2457:
----------------------------------
Component/s: Query Engine
(was: JDBC Connector)
> Assertion error with INTERSECT
> ------------------------------
>
> Key: TEIID-2457
> URL: https://issues.jboss.org/browse/TEIID-2457
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 7.7.6
> Reporter: Van Halbert
> Assignee: Steven Hawkins
>
> A sample query which produced the error is: SELECT IntKey FROM BQT1.SmallA INTERSECT SELECT IntNum FROM BQT1.SmallB ORDER BY INTKEY
> How to reproduce: Run the query using a client such as Squirrel multiple times. The results varies from returning an Assertion to returning actual results. In my test I ran the query five times and it returned actual results twice and the Assertion 3 times. The Assertion can be seen below.
> 2013-04-03 16:00:42,190 ERROR [org.teiid.PROCESSOR] (Worker4_QueryProcessorQueue146) Unexpected exception for request JyDs6MZ6d9zG.24
> java.lang.AssertionError: ASSERTION FAILED: expected reference to be not null
> at org.teiid.core.util.Assertion.failed(Assertion.java:73)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:100)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:92)
> at org.teiid.common.buffer.TupleBuffer.getBatch(TupleBuffer.java:217)
> at org.teiid.common.buffer.TupleBuffer$1.getBatch(TupleBuffer.java:331)
> at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:61)
> at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91)
> at org.teiid.query.processor.relational.MergeJoinStrategy.process(MergeJoinStrategy.java:163)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:212)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
> at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70)
> at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
> at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:85)
> at org.teiid.common.buffer.AbstractTupleSource.nextTuple(AbstractTupleSource.java:48)
> at org.teiid.query.processor.relational.SortUtility.initialSort(SortUtility.java:248)
> at org.teiid.query.processor.relational.SortUtility.sort(SortUtility.java:185)
> at org.teiid.query.processor.relational.SortNode.sortPhase(SortNode.java:99)
> at org.teiid.query.processor.relational.SortNode.nextBatchDirect(SortNode.java:88)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
> at org.teiid.query.processor.relational.LimitNode.nextBatchDirect(LimitNode.java:100)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:244)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2381) Expanded source hint support
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2381?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2381:
---------------------------------------
Mark -
> Other than disambiguating the current syntax as a hint that properly belongs on the Query object rather than the first column object, I think it's separate from the other issues.
It's two-fold, the primary concern is the placement on subqueries, union branches, with clause items, etc. The secondary is for sources that support more complicated positional hints, such as a column hint. The primary is fairly feasible. Arbitrary positions however are currently not.
> don't understand your third point or I'm confusing your first and third points. Maybe provide an example illustrating this?
The first and the third are somewhat related. If you restrict the first to only full queries that presumably are not able to be pushed with the parent, then they can all be associated with their respective push-down. If you want a hint to appear in a push-down subqueries, that would require additional translator support - e.g. a query and its subquery both have hints and are pushed together - "SELECT /\*+ sh ... \*/ ... (SELECT /\*+ sh ... \*/ ... ) FROM ..." such that the pushdown has both hints.
> the propagation rules can be straightforward
Ignoring optimization yes. However the optimizer free for example to remove view layers. For example if you have two views each with their own source hint and you join them together. The optimizer removes the view layers and reorders the join. Where do the hints apply and why?
> I would simply add public Iteratable<String> getSourceHints() to the API
That is one possibility. The optimizer could just collect all seemingly applicable hints and leave it up to the translator to decide.
> Expanded source hint support
> ----------------------------
>
> Key: TEIID-2381
> URL: https://issues.jboss.org/browse/TEIID-2381
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Attachments: HintsExecutionFactory.java, RandomExecutionFactory.java, RandomNumberStoredProcedureExecution.java, SysviewHintsExecutionTest.java
>
>
> We currently look at the source hint in only the root user query (not in subqueries nor the with clause) and only consider it in a very narrow set of circumstances when it's used in a view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2459) WITH statement translated wrongly for Oracle 10
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-2459?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration updated TEIID-2459:
-------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=908004
> WITH statement translated wrongly for Oracle 10
> ------------------------------------------------
>
> Key: TEIID-2459
> URL: https://issues.jboss.org/browse/TEIID-2459
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 7.7.6
> Reporter: Filip Nguyen
> Assignee: Steven Hawkins
> Fix For: 8.4
>
>
> I am using Oracle 10g [1] and I have set the Oracle version to 9.2 in a dynamic VDB [2].
> I am running simple WITH statement against Oracle: WITH x AS (select DATEVALUE from SMALLA) select DATEVALUE from x. Which throws [3]. Probably because Oracle 10 doesn't support column aliasing "WITH x (DATEVALUE)".
> [1] Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Production
> With the Partitioning, OLAP and Data Mining options
> [2] <property name="DatabaseVersion" value="9.2" />
> [3] Caused by: org.teiid.translator.TranslatorException: Error Code:32033 Message:Remote org.teiid.translator.jdbc.JDBCExecutionException: Error Code:32033 Message:'ORA-32033: unsupported column aliasing
> ' error executing statement(s): [Prepared Values: [] SQL: WITH x (DATEVALUE) AS (SELECT g_0."DATEVALUE" FROM "BQT2_RO"."SMALLA" g_0) SELECT g_0.DATEVALUE FROM x g_0]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2381) Expanded source hint support
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2381?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2381:
---------------------------------------
Van - that is one of the bullets above.
> Expanded source hint support
> ----------------------------
>
> Key: TEIID-2381
> URL: https://issues.jboss.org/browse/TEIID-2381
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Attachments: HintsExecutionFactory.java, RandomExecutionFactory.java, RandomNumberStoredProcedureExecution.java, SysviewHintsExecutionTest.java
>
>
> We currently look at the source hint in only the root user query (not in subqueries nor the with clause) and only consider it in a very narrow set of circumstances when it's used in a view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2381) Expanded source hint support
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-2381?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-2381:
------------------------------------
Can / will this enhancement cover the ability to add multiple source hints on WITH (PRODMGT-324)?
> Expanded source hint support
> ----------------------------
>
> Key: TEIID-2381
> URL: https://issues.jboss.org/browse/TEIID-2381
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Attachments: HintsExecutionFactory.java, RandomExecutionFactory.java, RandomNumberStoredProcedureExecution.java, SysviewHintsExecutionTest.java
>
>
> We currently look at the source hint in only the root user query (not in subqueries nor the with clause) and only consider it in a very narrow set of circumstances when it's used in a view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months