[JBoss JIRA] (TEIID-2605) Optimization substitutes wrong column in where clause
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2605?page=com.atlassian.jira.plugin... ]
Steven Hawkins closed TEIID-2605.
---------------------------------
> Optimization substitutes wrong column in where clause
> -----------------------------------------------------
>
> Key: TEIID-2605
> URL: https://issues.jboss.org/browse/TEIID-2605
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.1
> Environment: z/OS
> Reporter: Jeff Hayes
> Assignee: Steven Hawkins
> Attachments: query_plan.txt, views.xml
>
>
> Optimization results in a query with a different column in the WHERE clause producing an empty result set.
> Full query plan is attached but the beginning and ending queries are shown below. Note that the subject column of the IN clause is SCOPEID but optimization changes it to AUTHID for some reason.
> USER COMMAND:
> SELECT * FROM SECURITY.SCPXREF AS CHORUS_B WHERE (CHORUS_B.SYSID = 'DE29') AND ((CHORUS_B.SCOPEID IN (SELECT SN5.SCOPEID FROM SECURI
> TY.SCPNEXT AS SN5 WHERE (SYSID = 'DE29') AND (NEXTREC IN (SELECT SN4.SCOPEID FROM SECURITY.SCPNEXT AS SN4 WHERE (SYSID = 'DE29') AND
> (NEXTREC IN (SELECT SN3.SCOPEID FROM SECURITY.SCPNEXT AS SN3 WHERE (SYSID = 'DE29') AND (NEXTREC IN (SELECT SN2.SCOPEID FROM SECURI
> TY.SCPNEXT AS SN2 WHERE (SYSID = 'DE29') AND (NEXTREC = 'CHRDEPT1'))))))))) OR (CHORUS_B.SCOPEID IN (SELECT SN4.SCOPEID FROM SECURIT
> Y.SCPNEXT AS SN4 WHERE (SYSID = 'DE29') AND (NEXTREC IN (SELECT SN3.SCOPEID FROM SECURITY.SCPNEXT AS SN3 WHERE (SYSID = 'DE29') AND
> (NEXTREC IN (SELECT SN2.SCOPEID FROM SECURITY.SCPNEXT AS SN2 WHERE (SYSID = 'DE29') AND (NEXTREC = 'CHRDEPT1'))))))) OR (CHORUS_B.SC
> OPEID IN (SELECT SN3.SCOPEID FROM SECURITY.SCPNEXT AS SN3 WHERE (SYSID = 'DE29') AND (NEXTREC IN (SELECT SN2.SCOPEID FROM SECURITY.S
> CPNEXT AS SN2 WHERE (SYSID = 'DE29') AND (NEXTREC = 'CHRDEPT1'))))) OR (CHORUS_B.SCOPEID IN (SELECT SN2.SCOPEID FROM SECURITY.SCPNEX
> T AS SN2 WHERE (SYSID = 'DE29') AND (NEXTREC = 'CHRDEPT1'))) OR (CHORUS_B.SCOPEID = 'CHRDEPT1'))
> OPTIMIZATION COMPLETE:
> PROCESSOR PLAN:
> AccessNode(10) output=[x.sysid AS sysid, x.scopeid AS authid, x.authid AS scopeid, x.authtype AS authtype] SELECT g_0.SYSID, g_0.SCO
> PEID, g_0.AUTHID, g_0.AUTHTYPE FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPXREF AS g_0 WHERE (g_0.SYSID = 'DE29') AND ((g_0.AUTHID IN
> (SELECT g_1.SCOPEID FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_1 WHERE (g_1.SYSID = 'DE29') AND (g_1.NEXTREC IN (SELECT g_2
> .SCOPEID FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_2 WHERE (g_2.SYSID = 'DE29') AND (g_2.NEXTREC IN (SELECT g_3.SCOPEID FR
> OM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_3 WHERE (g_3.SYSID = 'DE29') AND (g_3.NEXTREC IN (SELECT g_4.SCOPEID FROM SECURITY
> _CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_4 WHERE (g_4.SYSID = 'DE29') AND (g_4.NEXTREC = 'CHRDEPT1'))))))))) OR (g_0.AUTHID IN (SELECT
> g_5.SCOPEID FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_5 WHERE (g_5.SYSID = 'DE29') AND (g_5.NEXTREC IN (SELECT g_6.SCOPEI
> D FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_6 WHERE (g_6.SYSID = 'DE29') AND (g_6.NEXTREC IN (SELECT g_7.SCOPEID FROM SECU
> RITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_7 WHERE (g_7.SYSID = 'DE29') AND (g_7.NEXTREC = 'CHRDEPT1'))))))) OR (g_0.AUTHID IN (SELE
> CT g_8.SCOPEID FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_8 WHERE (g_8.SYSID = 'DE29') AND (g_8.NEXTREC IN (SELECT g_9.SCOP
> EID FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_9 WHERE (g_9.SYSID = 'DE29') AND (g_9.NEXTREC = 'CHRDEPT1'))))) OR (g_0.AUTH
> ID IN (SELECT g_10.SCOPEID FROM SECURITY_CIA_DB2_D91BPTIB_CIADB01.SCPNEXT AS g_10 WHERE (g_10.SYSID = 'DE29') AND (g_10.NEXTREC = 'C
> HRDEPT1'))) OR (g_0.AUTHID = 'CHRDEPT1'))
> The view definitions are shown below:
> <view name="SCPNEXT">
> <columns>
> <column name="sysid" type="varchar"/>
> <column name="scopeid" type="varchar"/>
> <column name="nextrec" type="varchar"/>
> </columns>
> <definition>
> #if ($db.count("select count(*) from sys.tables where Name='config' and SchemaName = 'security_db'") > 0)
> #set ($count = 0)
> #foreach($t in $db.query("select DB_TYPE, DB_LOC, DB_QUAL from security_db.config WHERE TYPE = 'CIA'"))
> #if ($db.count("select count(*) from sys.tables where SchemaName = 'SECURITY_CIA_${t.DB_TYPE}_${t.DB_LOC}_${t.DB_QUAL}'") == 0)
> #set ($count = $count + 1)
> #end
> #end
> #if ($count == 0)
> SELECT t.sysid, t.scopeid, t.nextrec
> FROM (
> #foreach($t in $db.query("select DB_TYPE, DB_LOC, DB_QUAL from security_db.config WHERE TYPE = 'CIA'"))
> SELECT n.sysid, n.scopeid, n.nextrec
> FROM SECURITY_CIA_${t.DB_TYPE}_${t.DB_LOC}_${t.DB_QUAL}.SCPNEXT n
> #if( $velocityHasNext ) UNION #end
> #end
> ) AS t
> #end
> #end
> </definition>
> </view>
> <view name="SCPXREF">
> <columns>
> <column name="sysid" type="varchar"/>
> <column name="authid" type="varchar"/>
> <column name="scopeid" type="varchar"/>
> <column name="authtype" type="varchar"/>
> </columns>
> <definition>
> #if ($db.count("select count(*) from sys.tables where Name='config' and SchemaName = 'security_db'") > 0)
> #set ($count = 0)
> #foreach($t in $db.query("select DB_TYPE, DB_LOC, DB_QUAL from security_db.config WHERE TYPE = 'CIA'"))
> #if ($db.count("select count(*) from sys.tables where SchemaName = 'SECURITY_CIA_${t.DB_TYPE}_${t.DB_LOC}_${t.DB_QUAL}'") == 0)
> #set ($count = $count + 1)
> #end
> #end
> #if ($count == 0)
> SELECT sysid, scopeid, authid, authtype
> FROM (
> #foreach($t in $db.query("select DB_TYPE, DB_LOC, DB_QUAL from security_db.config WHERE TYPE = 'CIA'"))
> SELECT x.sysid, x.scopeid, x.authid, x.authtype
> FROM SECURITY_CIA_${t.DB_TYPE}_${t.DB_LOC}_${t.DB_QUAL}.SCPXREF x
> #if( $velocityHasNext ) UNION #end
> #end
> ) AS t
> #end
> #end
> </definition>
> </view>
> I guess the best way to test this is to define these views and run the input query with SHOWPLAN=DEBUG and see if the AUTHID substitution is occurring.
> Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
22 hours, 17 minutes
[JBoss JIRA] (TEIID-3748) Impala translator - SELECT and HAVING statements are translating differently for Case statements
by Don Krapohl (JIRA)
[ https://issues.jboss.org/browse/TEIID-3748?page=com.atlassian.jira.plugin... ]
Don Krapohl commented on TEIID-3748:
------------------------------------
We are using the default translator. Any other logs I can provide?
> Impala translator - SELECT and HAVING statements are translating differently for Case statements
> ------------------------------------------------------------------------------------------------
>
> Key: TEIID-3748
> URL: https://issues.jboss.org/browse/TEIID-3748
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 8.11.4
> Environment: Ubuntu Trusty
> Reporter: Don Krapohl
> Assignee: Steven Hawkins
> Labels: Impala_Translator, Translators
> Fix For: 8.12.2
>
> Attachments: server.log
>
>
> Error from Impala-
> all DISTINCT aggregate functions need to have the same set of parameters as count(DISTINCT (CASE WHEN (secondcol >= 0) THEN 1 ELSE CAST(NULL AS STRING) END))
> deviating function: count(DISTINCT (CASE WHEN (secondcol >= 0) THEN 1 ELSE NULL END))
> Query:
> SELECT user_key, sum(firstcol),count(distinct case when secondcol >= 0 then 1 end)
> FROM sometable
> WHERE customer_key=6
> GROUP BY user_key
> HAVING sum(firstcol)>100
> AND count(distinct case when secondcol >= 0 then 1 end)=0
>
> Query explanation:
> For all users
> Add up values in the firstcol column (integer column)
> count distinct values in secondcol where secondcol value zero or more
> otherwise return null (output is string)
> Translated Teiid query:
> SELECT user_key, SUM(firstcol) as `EXPR_0`, COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' ELSE CAST(NULL AS STRING) END)) as `EXPR_1`
> FROM sometable
> WHERE customer_key` = 6
> HAVING (EXPR_0 > 100) AND (COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' ELSE NULL END)) = 0))
> Note the difference between the select and having for EXPR_1:
> Select - THEN '1' ELSE CAST(NULL AS STRING) END
> Having - THEN '1' ELSE NULL END
> Impala doesn't accept that these are the same aggregate function. Aliases aren't accepted in the HAVING.
> One further observation- if we swap the translation and write the statement in the select as
> COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' *ELSE NULL END*))
> Teiid translates the SELECT to
> COUNT(DISTINCT (CASE WHEN (secondcol >= 0) THEN '1' *ELSE CAST(NULL AS STRING) END*))
> So it always makes these mismatched.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (TEIID-3314) Add translator for Elasticsearch
by Tom Arnold (JIRA)
[ https://issues.jboss.org/browse/TEIID-3314?page=com.atlassian.jira.plugin... ]
Tom Arnold commented on TEIID-3314:
-----------------------------------
Van, after looking into this a little bit more I agree the REST would be a better approach than using the Java client.
We put together a quick internal prototype using the Java client and there were a couple of downsides.
* No way to do authentication. Most people seem to put ES behind Apache/Nginx to add basic auth, but this is not possible using the Java client (at least, not without using an addon like Shield).
* The module was 40-50mb because it included Lucene. It's possible that this could be stripped down slightly.
* Unclear if this approach would be portable between different server versions, since the Java client is essentially another server node in the cluster AFAICT.
We used the mappings API for metadata. One issue we ran into here is that ES does not distinguish between scalar and array values, so handling these fields automatically is difficult. TEIID-3349 might help here, or perhaps taking an approach similar to the Accumulo translator (query first record and then parse metadata from that).
> Add translator for Elasticsearch
> --------------------------------
>
> Key: TEIID-3314
> URL: https://issues.jboss.org/browse/TEIID-3314
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Tom Arnold
> Labels: spatial
> Fix For: Open To Community
>
>
> Elasticseach is an open source search server based on Lucene.
> Use case would be similar to that of SOLR. There is a (slightly outdated) comparsion between the two here: http://solr-vs-elasticsearch.com/.
> Java API docs are here: http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/curre...
> Filter DSL documentation is here: http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/curre...
> JBoss EAP 6 module docs: http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/curre...
> It supports several types of spatial filters:
> * Bounding box
> * Polygon
> * Shape - This is based on JTS; I think the difference between this and Polygon is this allows options like WITHIN/INTERSECTS/DISJOINT.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (TEIID-3797) Use the term local consistently
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-3797:
-------------------------------------
Summary: Use the term local consistently
Key: TEIID-3797
URL: https://issues.jboss.org/browse/TEIID-3797
Project: Teiid
Issue Type: Quality Risk
Components: Server
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 9.0
To not have confusion with Teiid embedded, we should use the term local consistently to refer to the in-vm connections. This means updating the docs/config and renaming the embeddedprofile to localprofile.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (TEIID-3671) filter on child table not working
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3671?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-3671.
-----------------------------------
Resolution: Done
Thanks Juraj.
> filter on child table not working
> ---------------------------------
>
> Key: TEIID-3671
> URL: https://issues.jboss.org/browse/TEIID-3671
> Project: Teiid
> Issue Type: Bug
> Components: OData
> Reporter: Prashanthi Kairuppala
> Assignee: Johnathon Lee
> Priority: Critical
> Fix For: 8.7.5, 8.11.4, 8.12
>
> Attachments: parent_childTables.PNG
>
>
>
> Please find the screenshot attached which has id as a foreign key. when i give the url - //localhost:8080/odata/testVDB/sales('13')/sales_uk?$format=json, i get the results from sales_uk table which are having id=13.
>
> Now i am trying to apply a filter on this url, i am supposed to get results which are having id=13 and satisfy the filter condition, but this url blindly returns me data from sales_uk table which satisfy the filter condition. URL- //localhost:8080/odata/testVDB/sales('13')/sales_uk?$format=json&$filter=DISCOUNT eq '22'
>
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (TEIID-3671) filter on child table not working
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3671?page=com.atlassian.jira.plugin... ]
Steven Hawkins closed TEIID-3671.
---------------------------------
> filter on child table not working
> ---------------------------------
>
> Key: TEIID-3671
> URL: https://issues.jboss.org/browse/TEIID-3671
> Project: Teiid
> Issue Type: Bug
> Components: OData
> Reporter: Prashanthi Kairuppala
> Assignee: Johnathon Lee
> Priority: Critical
> Fix For: 8.12, 8.11.4, 8.7.5
>
> Attachments: parent_childTables.PNG
>
>
>
> Please find the screenshot attached which has id as a foreign key. when i give the url - //localhost:8080/odata/testVDB/sales('13')/sales_uk?$format=json, i get the results from sales_uk table which are having id=13.
>
> Now i am trying to apply a filter on this url, i am supposed to get results which are having id=13 and satisfy the filter condition, but this url blindly returns me data from sales_uk table which satisfy the filter condition. URL- //localhost:8080/odata/testVDB/sales('13')/sales_uk?$format=json&$filter=DISCOUNT eq '22'
>
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (TEIID-3671) filter on child table not working
by Juraj Duráni (JIRA)
[ https://issues.jboss.org/browse/TEIID-3671?page=com.atlassian.jira.plugin... ]
Juraj Duráni commented on TEIID-3671:
-------------------------------------
I think it should be fixed. You can close this JIRA.
> filter on child table not working
> ---------------------------------
>
> Key: TEIID-3671
> URL: https://issues.jboss.org/browse/TEIID-3671
> Project: Teiid
> Issue Type: Bug
> Components: OData
> Reporter: Prashanthi Kairuppala
> Assignee: Johnathon Lee
> Priority: Critical
> Fix For: 8.12, 8.11.4, 8.7.5
>
> Attachments: parent_childTables.PNG
>
>
>
> Please find the screenshot attached which has id as a foreign key. when i give the url - //localhost:8080/odata/testVDB/sales('13')/sales_uk?$format=json, i get the results from sales_uk table which are having id=13.
>
> Now i am trying to apply a filter on this url, i am supposed to get results which are having id=13 and satisfy the filter condition, but this url blindly returns me data from sales_uk table which satisfy the filter condition. URL- //localhost:8080/odata/testVDB/sales('13')/sales_uk?$format=json&$filter=DISCOUNT eq '22'
>
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (TEIID-3635) Auto configuration of buffermanager values can be incorrect
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3635?page=com.atlassian.jira.plugin... ]
Steven Hawkins closed TEIID-3635.
---------------------------------
> Auto configuration of buffermanager values can be incorrect
> -----------------------------------------------------------
>
> Key: TEIID-3635
> URL: https://issues.jboss.org/browse/TEIID-3635
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.0
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 8.12, 8.11.3, 8.7.5
>
>
> If the heap size is significantly larger than 2 gb, such that the auto computed value for the max reserve is larger than 2 gb and the memory buffer size is also auto calculated, then the computed values for max reserve and max memory buffer can be significantly different than expected. At worst this results in the max memory buffer size of 0, which just default the buffer to being accessible by a single thread.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months