[JBoss JIRA] (TEIID-4455) Impala Translator - Change planning step for from_unixtime() pushdown
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4455?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4455:
----------------------------------
Fix Version/s: 9.2
Affects Version/s: (was: 8.13.x)
(was: 9.0.4)
> Impala Translator - Change planning step for from_unixtime() pushdown
> ---------------------------------------------------------------------
>
> Key: TEIID-4455
> URL: https://issues.jboss.org/browse/TEIID-4455
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector
> Reporter: Don Krapohl
> Assignee: Steven Hawkins
> Priority: Minor
> Labels: Impala_Translator
> Fix For: 9.2
>
>
> Impala and Teiid both have from_unixtime() functions and per the forum reference the mapping to timestampadd() appears to come before the test for pushdown support of the impala native function.
> As a query request we require the impala from_unixtime() function be executed instead of the Teiid-native version.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 6 months
[JBoss JIRA] (TEIID-4456) Enable the abiltiy to support nested and non-nested message descriptors
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4456?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4456:
----------------------------------
Fix Version/s: 9.1
Affects Version/s: (was: 9.2)
> Enable the abiltiy to support nested and non-nested message descriptors
> -----------------------------------------------------------------------
>
> Key: TEIID-4456
> URL: https://issues.jboss.org/browse/TEIID-4456
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Van Halbert
> Fix For: 9.1
>
>
> Protobuf definition files can have messages defined in a nested and non-nested form.
> Nested:
> {code}
> package bigdata;
> message DataEntity {
> ..
> optional MetaData context = 6;
> message MetaData {
> ...
> }
> }
> {code}
> Non-nested:
> {code}
> package bigdata;
> message DataEntity {
> ..
> optional MetaData context = 6;
> }
> message MetaData {
> ...
> }
> {code}
> The ProtobufMetadataProcessor needs to be changed to so that it can create the metadata from either use case.
> Currently it obtains the child descriptor messages from the parent descriptor.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 6 months
[JBoss JIRA] (TEIID-4452) Pushdown SQL/XML
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4452?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-4452:
-------------------------------------
Fix Version/s: 9.2
Assignee: (was: Steven Hawkins)
I realized I hadn't answered your question from before:
> Do you have an idea of what work would be involved? I can probably work on it if it doesn't fit on your roadmap.
The work at a high level would be to:
- Add relevant capabilities methods to the ExecutionFactory
- Allow those methods to be converted into Capability enums.
- Change the CriteriaCapabilityValidationVisitor (that's a legacy misnomer, as it should be ExpressionCapabilityValidationVisitor) to check the capabilities.
- Add org.teiid.language objects that represent the relevant sql/xml constructs - this is necessary as most of the sql/xml functions use special constructs that are forced fit for just the standard Function with a list of Expression arguments.
- Have the LanguageBridgeFactory translate into the translator language objects
- Add supporting logic in the translator api for visitation and SQL stringification
- Finally handle any translator specific modification for the generated SQL.
As for the roadmap, we can tentatively look at this for 9.2, but it won't be a high priority.
> Pushdown SQL/XML
> ----------------
>
> Key: TEIID-4452
> URL: https://issues.jboss.org/browse/TEIID-4452
> Project: Teiid
> Issue Type: Enhancement
> Components: JDBC Connector
> Affects Versions: 9.x
> Reporter: Tom Arnold
> Fix For: 9.2
>
>
> It would be helpful to pushdown SQL/XML functions for JDBC databases that support them.
> Databases:
> - Oracle
> - Postgres
> Functions:
> - XMLELEMENT
> - XMLATTRIBUTES
> - XMLAGG
> - XMLCONCAT
> We have an application that uses these functions and we support Oracle, Postgres, and Teiid (to integrate external data sources). Here's an example query generated by this application:
> {code:xml}
> select
> "ITEMS"."ITEM_ID" "ITEM_ID",
> (
> select xmlelement (
> name "attributes",
> xmlagg(
> xmlelement(
> name "attribute",
> xmlattributes(
> "ITEM_ATTRIBUTES"."ATTR_NAME" as "name",
> "ITEM_ATTRIBUTES"."ATTR_VALUE" as "value"
> )
> )
> )
> )
> from "ITEM_ATTRIBUTES" "ITEM_ATTRIBUTES"
> where (
> "ITEMS"."ITEM_ID" = "ITEM_ATTRIBUTES"."ITEM_ID"
> )
> ) "ITEM_ATTRIBUTES"
> {code}
> {code}
> ITEM_ID ITEM_ATTRIBUTES
> 3 <attributes><attribute name="foo" value="bar"></attribute><attribute name="baz" value="blah"></attribute></attributes>
> 4 <attributes><attribute name="foo" value="123"></attribute><attribute name="baz" value="456"></attribute></attributes>
> {code}
> When we are using Teiid queries like this do not work well because they cannot be pushed down, so we get N-many queries (one per row) and the query ends up taking much longer than if performed directly against Oracle/Postgres.
> Since SQL/XML is standard and supported by both Oracle/Postgres it would be good if queries like this could be pushed in their entirety so that there would be no performance difference between going through Teiid and native.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 6 months
[JBoss JIRA] (TEIID-4467) Google translator throws exception invalid_query for GROUP BY
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4467?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-4467:
-------------------------------------
Component/s: Misc. Connectors
Fix Version/s: 9.2
Assignee: (was: Steven Hawkins)
To address this properly we'll have to add capabilities, but are only applicable to google and are not common query cases, so pushing to 9.2.
> Google translator throws exception invalid_query for GROUP BY
> --------------------------------------------------------------
>
> Key: TEIID-4467
> URL: https://issues.jboss.org/browse/TEIID-4467
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.12
> Reporter: Lucie Fabrikova
> Fix For: 9.2
>
> Attachments: googlespreadsheet-vdb.xml, server.log
>
>
> Google translator produces following exception for a query: "SELECT IntKey FROM bqt1.SmallA GROUP BY IntKey":
> 16:27:26,955 ERROR [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue14) TEIID30019 Unexpected exception for request vn+N1XY8Atf5.6: org.teiid.resource.adapter.google.common.SpreadsheetOperationException: [invalid_query]
> at org.teiid.resource.adapter.google.dataprotocol.GoogleDataProtocolAPI$DataProtocolQueryStrategy.executeAndParse(GoogleDataProtocolAPI.java:196)
> ...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 6 months
[JBoss JIRA] (TEIID-4477) LDAP connector disconnects intermittently with error "TEIID12002"
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4477?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4477.
-----------------------------------
Fix Version/s: 9.1
9.0.5
Resolution: Done
Updated the isAlive method so that the pool can properly remove stale connections when the validate-on-match or background-validation is configured. Also allowed the context factory to be set and added null checks to prevent the npe on close when execute fails.
> LDAP connector disconnects intermittently with error "TEIID12002"
> -----------------------------------------------------------------
>
> Key: TEIID-4477
> URL: https://issues.jboss.org/browse/TEIID-4477
> Project: Teiid
> Issue Type: Bug
> Components: LDAP Connector
> Affects Versions: 8.12.5
> Environment: Red Hat JBoss Data Virtualization - Version 6.3.0
> Reporter: Rafael Coutinho
> Assignee: Steven Hawkins
> Fix For: 9.1, 9.0.5
>
>
> We are using LDAP connector for virtualizing the ldap employee directory data. For some reason every 3 to 4 hours we get a connection error between JDV and LDAP.
> The exception says it could not find the context name, but after restarting it it works again.
> Here is the exception:
> 11:21:12,868 WARN [org.teiid.CONNECTOR] (Worker62_QueryProcessorQueue108177) Connector worker process failed for atomic-request=6S71mw5pkfvo.106.3.19906: org.teiid.translator.TranslatorException: TEIID12002 Failed to create LDAP search context from the specified context name ou=users,dc=redhat,dc=com. Check the table/group name in source to ensure the context exists.
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.createSearchContext(LDAPSyncQueryExecution.java:148) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.execute(LDAPSyncQueryExecution.java:130) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:364)
> at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:508) [rt.jar:2.6 (07-19-2016)]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy80.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:277) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:785) [vm.jar:1.8.0-internal]
> Caused by: javax.naming.CommunicationException: connection closed [Root exception is java.io.IOException: connection closed]; remaining name 'ou=users,dc=redhat,dc=com'
> at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2014) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapCtx.doSearchOnce(LdapCtx.java:1945) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapCtx.c_lookup(LdapCtx.java:1039) [rt.jar:1.8.0-internal]
> at com.sun.jndi.toolkit.ctx.ComponentContext.p_lookup(ComponentContext.java:554) [rt.jar:1.8.0-internal]
> at com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:189) [rt.jar:1.8.0-internal]
> at com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:178) [rt.jar:1.8.0-internal]
> at javax.naming.InitialContext.lookup(InitialContext.java:428) [rt.jar:1.8.0-internal]
> at javax.naming.InitialContext.lookup(InitialContext.java:428) [rt.jar:1.8.0-internal]
> at org.teiid.resource.adapter.ldap.LDAPConnectionImpl.lookup(LDAPConnectionImpl.java:193)
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.createSearchContext(LDAPSyncQueryExecution.java:146) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> ... 18 more
> Caused by: java.io.IOException: connection closed
> at com.sun.jndi.ldap.LdapClient.ensureOpen(LdapClient.java:1604) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:542) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1997) [rt.jar:1.8.0-internal]
> ... 27 more
> 11:21:12,871 ERROR [org.teiid.CONNECTOR] (Worker63_QueryProcessorQueue108178) null: java.lang.NullPointerException
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.close(LDAPSyncQueryExecution.java:186) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.close(ConnectorWorkItem.java:255)
> at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:508) [rt.jar:2.6 (07-19-2016)]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy80.close(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.fullyCloseSource(DataTierTupleSource.java:338)
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:394)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:161)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:391)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.SelectNode.nextBatchDirect(SelectNode.java:104)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:150)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.DupRemoveNode.nextBatchDirect(DupRemoveNode.java:62)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:145)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:164)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:472)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:348)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:785) [vm.jar:1.8.0-internal]
> 11:21:12,874 DEBUG [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (Worker63_QueryProcessorQueue108178) LDAPSource: returnConnection(6306ef86, false) [1/20]
> 11:21:12,874 WARN [org.teiid.PROCESSOR] (Worker63_QueryProcessorQueue108178) TEIID30020 Processing exception for request 6S71mw5pkfvo.106 'TEIID30504 local: TEIID12002 Failed to create LDAP search context from the specified context name ou=users,dc=redhat,dc=com. Check the table/group name in source to ensure the context exists.'. Originally TeiidProcessingException 'connection closed' LdapClient.java:1604. Enable more detailed logging to see the entire stacktrace.
> 11:21:16,120 DEBUG [org.apache.catalina.session] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Start expire sessions StandardManager at 1474903276120 sessioncount 0
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 6 months