[JBoss JIRA] (TEIID-3748) Impala translator - SELECT and HAVING statements are translating differently for Case statements
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3748?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-3748.
-----------------------------------
Fix Version/s: 9.0
8.13.1
Resolution: Done
Updated the rewriter logic to also rewrite aggregate arguments - regardless of where they appear in the query.
> Impala translator - SELECT and HAVING statements are translating differently for Case statements
> ------------------------------------------------------------------------------------------------
>
> Key: TEIID-3748
> URL: https://issues.jboss.org/browse/TEIID-3748
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 8.11.4
> Environment: Ubuntu Trusty
> Reporter: Don Krapohl
> Assignee: Steven Hawkins
> Labels: Impala_Translator, Translators
> Fix For: 9.0, 8.13.1
>
> Attachments: server.log
>
>
> Error from Impala-
> all DISTINCT aggregate functions need to have the same set of parameters as count(DISTINCT (CASE WHEN (secondcol >= 0) THEN 1 ELSE CAST(NULL AS STRING) END))
> deviating function: count(DISTINCT (CASE WHEN (secondcol >= 0) THEN 1 ELSE NULL END))
> Query:
> SELECT user_key, sum(firstcol),count(distinct case when secondcol >= 0 then 1 end)
> FROM sometable
> WHERE customer_key=6
> GROUP BY user_key
> HAVING sum(firstcol)>100
> AND count(distinct case when secondcol >= 0 then 1 end)=0
>
> Query explanation:
> For all users
> Add up values in the firstcol column (integer column)
> count distinct values in secondcol where secondcol value zero or more
> otherwise return null (output is string)
> Translated Teiid query:
> SELECT user_key, SUM(firstcol) as `EXPR_0`, COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' ELSE CAST(NULL AS STRING) END)) as `EXPR_1`
> FROM sometable
> WHERE customer_key` = 6
> HAVING (EXPR_0 > 100) AND (COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' ELSE NULL END)) = 0))
> Note the difference between the select and having for EXPR_1:
> Select - THEN '1' ELSE CAST(NULL AS STRING) END
> Having - THEN '1' ELSE NULL END
> Impala doesn't accept that these are the same aggregate function. Aliases aren't accepted in the HAVING.
> One further observation- if we swap the translation and write the statement in the select as
> COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' *ELSE NULL END*))
> Teiid translates the SELECT to
> COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' *ELSE CAST(NULL AS STRING) END*))
> So it always makes these mismatched.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (TEIID-3748) Impala translator - SELECT and HAVING statements are translating differently for Case statements
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3748?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-3748:
----------------------------------
Issue Type: Bug (was: Quality Risk)
I was able to reproduce this locally. With the initial query:
SELECT stringkey, sum(intnum),count(distinct case when floatnum >= 0 then 1 end) FROM smalla WHERE intkey=6 GROUP BY stringkey HAVING sum(intnum)>100 AND count(distinct case when floatnum >= 0 then 1 end)=0
The pushdown is:
SELECT g_0.stringkey, SUM(g_0.intnum),
COUNT(DISTINCT CASE WHEN g_0.floatnum >= 0.0 THEN 1 END)
FROM `smalla` AS g_0 WHERE g_0.intkey = 6
GROUP BY g_0.stringkey HAVING SUM(g_0.intnum) > 100 AND
COUNT(DISTINCT CASE WHEN g_0.floatnum >= convert(0, float) THEN 1 END) = 0
So the non-evaluated cast is then causing an issue for Impala
> Impala translator - SELECT and HAVING statements are translating differently for Case statements
> ------------------------------------------------------------------------------------------------
>
> Key: TEIID-3748
> URL: https://issues.jboss.org/browse/TEIID-3748
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 8.11.4
> Environment: Ubuntu Trusty
> Reporter: Don Krapohl
> Assignee: Steven Hawkins
> Labels: Impala_Translator, Translators
> Attachments: server.log
>
>
> Error from Impala-
> all DISTINCT aggregate functions need to have the same set of parameters as count(DISTINCT (CASE WHEN (secondcol >= 0) THEN 1 ELSE CAST(NULL AS STRING) END))
> deviating function: count(DISTINCT (CASE WHEN (secondcol >= 0) THEN 1 ELSE NULL END))
> Query:
> SELECT user_key, sum(firstcol),count(distinct case when secondcol >= 0 then 1 end)
> FROM sometable
> WHERE customer_key=6
> GROUP BY user_key
> HAVING sum(firstcol)>100
> AND count(distinct case when secondcol >= 0 then 1 end)=0
>
> Query explanation:
> For all users
> Add up values in the firstcol column (integer column)
> count distinct values in secondcol where secondcol value zero or more
> otherwise return null (output is string)
> Translated Teiid query:
> SELECT user_key, SUM(firstcol) as `EXPR_0`, COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' ELSE CAST(NULL AS STRING) END)) as `EXPR_1`
> FROM sometable
> WHERE customer_key` = 6
> HAVING (EXPR_0 > 100) AND (COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' ELSE NULL END)) = 0))
> Note the difference between the select and having for EXPR_1:
> Select - THEN '1' ELSE CAST(NULL AS STRING) END
> Having - THEN '1' ELSE NULL END
> Impala doesn't accept that these are the same aggregate function. Aliases aren't accepted in the HAVING.
> One further observation- if we swap the translation and write the statement in the select as
> COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' *ELSE NULL END*))
> Teiid translates the SELECT to
> COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' *ELSE CAST(NULL AS STRING) END*))
> So it always makes these mismatched.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (TEIID-3826) Upgrade to use resteasy-jaxrs:jar:2.3.10.Final
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-3826?page=com.atlassian.jira.plugin... ]
Ramesh Reddy resolved TEIID-3826.
---------------------------------
Resolution: Done
Instead of creating a new common module, I just pulled in the single class into odata. Otherwise I could pull into "common" but that will introduce the dependency on Servlet api jar. I did not feel it was worth the effort.
> Upgrade to use resteasy-jaxrs:jar:2.3.10.Final
> ----------------------------------------------
>
> Key: TEIID-3826
> URL: https://issues.jboss.org/browse/TEIID-3826
> Project: Teiid
> Issue Type: Task
> Components: OData
> Affects Versions: 8.12
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Priority: Blocker
> Fix For: 8.12.5
>
>
> 8.12.x is currently dependent on resteasy-jaxrs:jar:2.3.7.Final
> In upgrading the IP-BOM to 30, which is dependent on 2.3.10, it seems the following class has been removed from javax-rs 2.3.10:
> org/jboss/resteasy/util/HttpServletRequestDelegate.java
> which breaks ProxyHttpServletRequest.java
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (TEIID-3949) Subqueries are wrongly pushed down to the underlying datasource
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3949?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-3949.
-----------------------------------
Fix Version/s: 8.12.5
8.13.1
Resolution: Done
Updated the logic to use the aggregates from the grouping node symbol map to ensure that the pre-evaluation flag is properly set.
> Subqueries are wrongly pushed down to the underlying datasource
> ---------------------------------------------------------------
>
> Key: TEIID-3949
> URL: https://issues.jboss.org/browse/TEIID-3949
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12
> Reporter: Salvatore R
> Assignee: Steven Hawkins
> Fix For: 9.0, 8.12.5, 8.13.1
>
>
> I have two schemas, one in PostgreSQL and another one in MySQL, called "pg" and "my" respectively.
> I defined the following tables in PG:
> {code:sql}
> CREATE TABLE test_a (a integer, b integer);
> CREATE TABLE test_only_pg (a integer, b integer);
> {code}
> and this table in MySQL:
> {code:sql}
> CREATE TABLE `test_b` (
> `b` int(11),
> `c` int(11)
> )
> {code}
> When I run this query:
> {code:sql}
> SELECT
> SUM(x.b - (SELECT a FROM pg.test_only_pg WHERE b = 1))
> FROM my.test_b x
> INNER JOIN pg.test_a y ON x.b = y.b
> {code}
> I get the following exception:
> {code:sql}
> 16:44:54,776 WARN [org.teiid.CONNECTOR] (Worker2_QueryProcessorQueue28) QWfZdmacM17x Connector worker process failed for atomic-request=QWfZdmacM17x.12.7.4: org.teiid.translator.jdbc.JDBCExecutionException: 1146 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.`b` AS c_0, SUM((g_0.`b` - (SELECT g_1."a" AS c_0 FROM "public"."test_only_pg" AS g_1 WHERE g_1."b" = 1 LIMIT 2))) AS c_1 FROM `test_tables`.`test_b` AS g_0 GROUP BY g_0.`b` ORDER BY c_0]
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:131) [translator-jdbc-8.12.4.jar:8.12.4]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:356)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_67]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_67]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_67]
> at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_67]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_67]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_67]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_67]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_67]
> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'public.test_only_pg' doesn't exist
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_67]
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_67]
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_67]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) [rt.jar:1.7.0_67]
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> at com.mysql.jdbc.Util.getInstance(Util.java:386)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4096)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2490)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2734)
> at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
> at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462)
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:123) [translator-jdbc-8.12.4.jar:8.12.4]
> ... 18 more
> {code}
> I could reproduce this behavior in Teiid 8.12.4.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (TEIID-3351) Quick Start "dynamicvdb-datafederation" needlessly made complicated
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-3351?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-3351:
------------------------------------
The materialization is being moved to its own quick start: dynamicvdb-materialization, along with the jdg-local-cache-materialization, and jdg-remote-cache-materialization.
> Quick Start "dynamicvdb-datafederation" needlessly made complicated
> -------------------------------------------------------------------
>
> Key: TEIID-3351
> URL: https://issues.jboss.org/browse/TEIID-3351
> Project: Teiid
> Issue Type: Quality Risk
> Components: Quick Starts
> Reporter: Ramesh Reddy
> Assignee: Van Halbert
> Fix For: 8.12.x
>
>
> The "dynamicvdb-datafederation" example in the Quick Starts example is needlessly made complicated. Originally this is designed to have one File and RDBMS source, to show a simple data integration through Teiid.
> Now, it has
> - Excel integration
> - Materialization Example
> - more models
> I am not opposed to having these features shown in an example, however not in this example. This needs to be as simple as possible to show a quick introduction to the Teiid. Please move these into separate quick starts.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (TEIID-3949) Subqueries are wrongly pushed down to the underlying datasource
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3949?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-3949:
----------------------------------
Component/s: Query Engine
Fix Version/s: 9.0
Affects Version/s: 8.12
The issue is that the pre-evaluation of the sub query is not happening inside the aggregate arguments. If the expression is pushed separately, the scalar value is correctly substituted.
> Subqueries are wrongly pushed down to the underlying datasource
> ---------------------------------------------------------------
>
> Key: TEIID-3949
> URL: https://issues.jboss.org/browse/TEIID-3949
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12
> Reporter: Salvatore R
> Assignee: Steven Hawkins
> Fix For: 9.0
>
>
> I have two schemas, one in PostgreSQL and another one in MySQL, called "pg" and "my" respectively.
> I defined the following tables in PG:
> {code:sql}
> CREATE TABLE test_a (a integer, b integer);
> CREATE TABLE test_only_pg (a integer, b integer);
> {code}
> and this table in MySQL:
> {code:sql}
> CREATE TABLE `test_b` (
> `b` int(11),
> `c` int(11)
> )
> {code}
> When I run this query:
> {code:sql}
> SELECT
> SUM(x.b - (SELECT a FROM pg.test_only_pg WHERE b = 1))
> FROM my.test_b x
> INNER JOIN pg.test_a y ON x.b = y.b
> {code}
> I get the following exception:
> {code:sql}
> 16:44:54,776 WARN [org.teiid.CONNECTOR] (Worker2_QueryProcessorQueue28) QWfZdmacM17x Connector worker process failed for atomic-request=QWfZdmacM17x.12.7.4: org.teiid.translator.jdbc.JDBCExecutionException: 1146 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.`b` AS c_0, SUM((g_0.`b` - (SELECT g_1."a" AS c_0 FROM "public"."test_only_pg" AS g_1 WHERE g_1."b" = 1 LIMIT 2))) AS c_1 FROM `test_tables`.`test_b` AS g_0 GROUP BY g_0.`b` ORDER BY c_0]
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:131) [translator-jdbc-8.12.4.jar:8.12.4]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:356)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_67]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_67]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_67]
> at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_67]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_67]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_67]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_67]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_67]
> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'public.test_only_pg' doesn't exist
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.7.0_67]
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [rt.jar:1.7.0_67]
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_67]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) [rt.jar:1.7.0_67]
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> at com.mysql.jdbc.Util.getInstance(Util.java:386)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4096)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2490)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2734)
> at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
> at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462)
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:123) [translator-jdbc-8.12.4.jar:8.12.4]
> ... 18 more
> {code}
> I could reproduce this behavior in Teiid 8.12.4.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (TEIID-2293) Update Object-Translator to add support for JPA/Hibernate search annotations in the metadata processor
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-2293?page=com.atlassian.jira.plugin... ]
Van Halbert edited comment on TEIID-2293 at 2/2/16 1:55 PM:
------------------------------------------------------------
I think it is. The reverse engineering of a table/view to a pojo will have 2 options:
- hibernate annotations (for JDG library mode)
- protobuf annotations (for JDG remote cache)
maybe the title should be adjusted.
was (Author: van.halbert):
I think it is. The reverse engineering of a table/view to a pojo will have 2 options:
- hibernate annotations (for JDG library mode)
- protobuf annotations (for JDG remote cache)
> Update Object-Translator to add support for JPA/Hibernate search annotations in the metadata processor
> ------------------------------------------------------------------------------------------------------
>
> Key: TEIID-2293
> URL: https://issues.jboss.org/browse/TEIID-2293
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.2
> Reporter: Van Halbert
> Assignee: Van Halbert
>
> Update Object-Translator to add support for JPA/Hibernate search annotations in the metadata processor. Coordinate with the JPA/OGM work that is being done.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month