[JBoss JIRA] (TEIID-4455) Impala Translator - Change planning step for from_unixtime() pushdown
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4455?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4455:
----------------------------------
Fix Version/s: 9.2
Affects Version/s: (was: 8.13.x)
(was: 9.0.4)
> Impala Translator - Change planning step for from_unixtime() pushdown
> ---------------------------------------------------------------------
>
> Key: TEIID-4455
> URL: https://issues.jboss.org/browse/TEIID-4455
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector
> Reporter: Don Krapohl
> Assignee: Steven Hawkins
> Priority: Minor
> Labels: Impala_Translator
> Fix For: 9.2
>
>
> Impala and Teiid both have from_unixtime() functions and per the forum reference the mapping to timestampadd() appears to come before the test for pushdown support of the impala native function.
> As a query request we require the impala from_unixtime() function be executed instead of the Teiid-native version.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 9 months
[JBoss JIRA] (TEIID-4456) Enable the abiltiy to support nested and non-nested message descriptors
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4456?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4456:
----------------------------------
Fix Version/s: 9.1
Affects Version/s: (was: 9.2)
> Enable the abiltiy to support nested and non-nested message descriptors
> -----------------------------------------------------------------------
>
> Key: TEIID-4456
> URL: https://issues.jboss.org/browse/TEIID-4456
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Van Halbert
> Fix For: 9.1
>
>
> Protobuf definition files can have messages defined in a nested and non-nested form.
> Nested:
> {code}
> package bigdata;
> message DataEntity {
> ..
> optional MetaData context = 6;
> message MetaData {
> ...
> }
> }
> {code}
> Non-nested:
> {code}
> package bigdata;
> message DataEntity {
> ..
> optional MetaData context = 6;
> }
> message MetaData {
> ...
> }
> {code}
> The ProtobufMetadataProcessor needs to be changed to so that it can create the metadata from either use case.
> Currently it obtains the child descriptor messages from the parent descriptor.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 9 months
[JBoss JIRA] (TEIID-4452) Pushdown SQL/XML
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4452?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-4452:
-------------------------------------
Fix Version/s: 9.2
Assignee: (was: Steven Hawkins)
I realized I hadn't answered your question from before:
> Do you have an idea of what work would be involved? I can probably work on it if it doesn't fit on your roadmap.
The work at a high level would be to:
- Add relevant capabilities methods to the ExecutionFactory
- Allow those methods to be converted into Capability enums.
- Change the CriteriaCapabilityValidationVisitor (that's a legacy misnomer, as it should be ExpressionCapabilityValidationVisitor) to check the capabilities.
- Add org.teiid.language objects that represent the relevant sql/xml constructs - this is necessary as most of the sql/xml functions use special constructs that are forced fit for just the standard Function with a list of Expression arguments.
- Have the LanguageBridgeFactory translate into the translator language objects
- Add supporting logic in the translator api for visitation and SQL stringification
- Finally handle any translator specific modification for the generated SQL.
As for the roadmap, we can tentatively look at this for 9.2, but it won't be a high priority.
> Pushdown SQL/XML
> ----------------
>
> Key: TEIID-4452
> URL: https://issues.jboss.org/browse/TEIID-4452
> Project: Teiid
> Issue Type: Enhancement
> Components: JDBC Connector
> Affects Versions: 9.x
> Reporter: Tom Arnold
> Fix For: 9.2
>
>
> It would be helpful to pushdown SQL/XML functions for JDBC databases that support them.
> Databases:
> - Oracle
> - Postgres
> Functions:
> - XMLELEMENT
> - XMLATTRIBUTES
> - XMLAGG
> - XMLCONCAT
> We have an application that uses these functions and we support Oracle, Postgres, and Teiid (to integrate external data sources). Here's an example query generated by this application:
> {code:xml}
> select
> "ITEMS"."ITEM_ID" "ITEM_ID",
> (
> select xmlelement (
> name "attributes",
> xmlagg(
> xmlelement(
> name "attribute",
> xmlattributes(
> "ITEM_ATTRIBUTES"."ATTR_NAME" as "name",
> "ITEM_ATTRIBUTES"."ATTR_VALUE" as "value"
> )
> )
> )
> )
> from "ITEM_ATTRIBUTES" "ITEM_ATTRIBUTES"
> where (
> "ITEMS"."ITEM_ID" = "ITEM_ATTRIBUTES"."ITEM_ID"
> )
> ) "ITEM_ATTRIBUTES"
> {code}
> {code}
> ITEM_ID ITEM_ATTRIBUTES
> 3 <attributes><attribute name="foo" value="bar"></attribute><attribute name="baz" value="blah"></attribute></attributes>
> 4 <attributes><attribute name="foo" value="123"></attribute><attribute name="baz" value="456"></attribute></attributes>
> {code}
> When we are using Teiid queries like this do not work well because they cannot be pushed down, so we get N-many queries (one per row) and the query ends up taking much longer than if performed directly against Oracle/Postgres.
> Since SQL/XML is standard and supported by both Oracle/Postgres it would be good if queries like this could be pushed in their entirety so that there would be no performance difference between going through Teiid and native.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 9 months
[JBoss JIRA] (TEIID-4467) Google translator throws exception invalid_query for GROUP BY
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4467?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-4467:
-------------------------------------
Component/s: Misc. Connectors
Fix Version/s: 9.2
Assignee: (was: Steven Hawkins)
To address this properly we'll have to add capabilities, but are only applicable to google and are not common query cases, so pushing to 9.2.
> Google translator throws exception invalid_query for GROUP BY
> --------------------------------------------------------------
>
> Key: TEIID-4467
> URL: https://issues.jboss.org/browse/TEIID-4467
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.12
> Reporter: Lucie Fabrikova
> Fix For: 9.2
>
> Attachments: googlespreadsheet-vdb.xml, server.log
>
>
> Google translator produces following exception for a query: "SELECT IntKey FROM bqt1.SmallA GROUP BY IntKey":
> 16:27:26,955 ERROR [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue14) TEIID30019 Unexpected exception for request vn+N1XY8Atf5.6: org.teiid.resource.adapter.google.common.SpreadsheetOperationException: [invalid_query]
> at org.teiid.resource.adapter.google.dataprotocol.GoogleDataProtocolAPI$DataProtocolQueryStrategy.executeAndParse(GoogleDataProtocolAPI.java:196)
> ...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 9 months
[JBoss JIRA] (TEIID-4477) LDAP connector disconnects intermittently with error "TEIID12002"
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4477?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4477.
-----------------------------------
Fix Version/s: 9.1
9.0.5
Resolution: Done
Updated the isAlive method so that the pool can properly remove stale connections when the validate-on-match or background-validation is configured. Also allowed the context factory to be set and added null checks to prevent the npe on close when execute fails.
> LDAP connector disconnects intermittently with error "TEIID12002"
> -----------------------------------------------------------------
>
> Key: TEIID-4477
> URL: https://issues.jboss.org/browse/TEIID-4477
> Project: Teiid
> Issue Type: Bug
> Components: LDAP Connector
> Affects Versions: 8.12.5
> Environment: Red Hat JBoss Data Virtualization - Version 6.3.0
> Reporter: Rafael Coutinho
> Assignee: Steven Hawkins
> Fix For: 9.1, 9.0.5
>
>
> We are using LDAP connector for virtualizing the ldap employee directory data. For some reason every 3 to 4 hours we get a connection error between JDV and LDAP.
> The exception says it could not find the context name, but after restarting it it works again.
> Here is the exception:
> 11:21:12,868 WARN [org.teiid.CONNECTOR] (Worker62_QueryProcessorQueue108177) Connector worker process failed for atomic-request=6S71mw5pkfvo.106.3.19906: org.teiid.translator.TranslatorException: TEIID12002 Failed to create LDAP search context from the specified context name ou=users,dc=redhat,dc=com. Check the table/group name in source to ensure the context exists.
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.createSearchContext(LDAPSyncQueryExecution.java:148) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.execute(LDAPSyncQueryExecution.java:130) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:364)
> at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:508) [rt.jar:2.6 (07-19-2016)]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy80.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:277) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:785) [vm.jar:1.8.0-internal]
> Caused by: javax.naming.CommunicationException: connection closed [Root exception is java.io.IOException: connection closed]; remaining name 'ou=users,dc=redhat,dc=com'
> at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2014) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapCtx.doSearchOnce(LdapCtx.java:1945) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapCtx.c_lookup(LdapCtx.java:1039) [rt.jar:1.8.0-internal]
> at com.sun.jndi.toolkit.ctx.ComponentContext.p_lookup(ComponentContext.java:554) [rt.jar:1.8.0-internal]
> at com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:189) [rt.jar:1.8.0-internal]
> at com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:178) [rt.jar:1.8.0-internal]
> at javax.naming.InitialContext.lookup(InitialContext.java:428) [rt.jar:1.8.0-internal]
> at javax.naming.InitialContext.lookup(InitialContext.java:428) [rt.jar:1.8.0-internal]
> at org.teiid.resource.adapter.ldap.LDAPConnectionImpl.lookup(LDAPConnectionImpl.java:193)
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.createSearchContext(LDAPSyncQueryExecution.java:146) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> ... 18 more
> Caused by: java.io.IOException: connection closed
> at com.sun.jndi.ldap.LdapClient.ensureOpen(LdapClient.java:1604) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:542) [rt.jar:1.8.0-internal]
> at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1997) [rt.jar:1.8.0-internal]
> ... 27 more
> 11:21:12,871 ERROR [org.teiid.CONNECTOR] (Worker63_QueryProcessorQueue108178) null: java.lang.NullPointerException
> at org.teiid.translator.ldap.LDAPSyncQueryExecution.close(LDAPSyncQueryExecution.java:186) [translator-ldap-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.close(ConnectorWorkItem.java:255)
> at sun.reflect.GeneratedMethodAccessor185.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:508) [rt.jar:2.6 (07-19-2016)]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy80.close(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.fullyCloseSource(DataTierTupleSource.java:338)
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:394)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:161)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:391)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.SelectNode.nextBatchDirect(SelectNode.java:104)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:150)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.DupRemoveNode.nextBatchDirect(DupRemoveNode.java:62)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:145)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:164)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:472)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:348)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:785) [vm.jar:1.8.0-internal]
> 11:21:12,874 DEBUG [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (Worker63_QueryProcessorQueue108178) LDAPSource: returnConnection(6306ef86, false) [1/20]
> 11:21:12,874 WARN [org.teiid.PROCESSOR] (Worker63_QueryProcessorQueue108178) TEIID30020 Processing exception for request 6S71mw5pkfvo.106 'TEIID30504 local: TEIID12002 Failed to create LDAP search context from the specified context name ou=users,dc=redhat,dc=com. Check the table/group name in source to ensure the context exists.'. Originally TeiidProcessingException 'connection closed' LdapClient.java:1604. Enable more detailed logging to see the entire stacktrace.
> 11:21:16,120 DEBUG [org.apache.catalina.session] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Start expire sessions StandardManager at 1474903276120 sessioncount 0
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 9 months
[JBoss JIRA] (TEIID-4479) Teiid Exception on multiple values criteria
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4479?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4479:
---------------------------------------
This looks to be the same as TEIID-4406 - can you verify it works with that fix?
> Teiid Exception on multiple values criteria
> -------------------------------------------
>
> Key: TEIID-4479
> URL: https://issues.jboss.org/browse/TEIID-4479
> Project: Teiid
> Issue Type: Bug
> Reporter: Mark Tawk
> Assignee: Steven Hawkins
>
> I'm using Teiid 9.0.0 with mysql
> I'm getting an exception on the query below when putting more then one value in the criteria : ("JPVPricedSecurities"."Reply_VendorID" IN (10, 2))
> if the criteria is on one value, the query executes with success;
> The case is specifically related to the query.
> {code}
> select SUM("JPVPricedSecurities"."ContractLine_Cost") as
> "JPVPricedSecurities_Contract",
> year("SRequestPerVendor"."SRequest_DateTime") as
> "DAYOFMONTHSRequestPerVendor_"
> from "DataMarketBusinessModel"."SRequestPerVendor" "SRequestPerVendor"
> LEFT JOIN "DataMarketBusinessModel"."JPVPricedSecurities" "JPVPricedSecurities" ON "SRequestPerVendor"."SRequest_AssetID" = "JPVPricedSecurities"."Reply_AssetID"
> LEFT JOIN "datamarketModel_MM"."datamarket"."Vendor" "Vendor" ON "JPVPricedSecurities"."Reply_VendorID" = "Vendor"."VendorID"
> where ("JPVPricedSecurities"."Reply_VendorID" IN (10, 2) )
> AND ("Vendor"."Vendor" IS NOT null )
> group by
> year("SRequestPerVendor"."SRequest_DateTime")
> {code}
> Here is the exception stack:
> {code}
> Daemon Thread [http-bio-8080-exec-2] (Suspended (exception NullPointerException))
> owns: Object (id=23929)
> owns: SocketWrapper<E> (id=23848)
> LanguageBridgeFactory.translate(DependentSetCriteria) line: 426
> LanguageBridgeFactory.translate(Criteria) line: 418
> LanguageBridgeFactory.translate(CompoundCriteria) line: 494
> LanguageBridgeFactory.translate(Criteria) line: 402
> LanguageBridgeFactory.translate(Query) line: 281
> LanguageBridgeFactory.translate(QueryCommand) line: 226
> LanguageBridgeFactory.translate(SubqueryFromClause) line: 386
> LanguageBridgeFactory.translate(FromClause) line: 344
> LanguageBridgeFactory.translate(Query) line: 276
> LanguageBridgeFactory.translate(Command) line: 184
> ConnectorWorkItem.<init>(AtomicRequestMessage, ConnectorManager) line: 175
> EmbeddedServer$ProviderAwareConnectorManagerRepository$1(ConnectorManager).registerRequest(AtomicRequestMessage) line: 203
> DataTierManagerImpl.registerRequest(CommandContext, Command, String, RegisterRequestParameter) line: 1011
> TempTableDataManager.registerRequest(CommandContext, Command, String, RegisterRequestParameter) line: 179
> DependentAccessNode(AccessNode).registerRequest(Command) line: 500
> DependentAccessNode(AccessNode).openInternal() line: 268
> DependentAccessNode(AccessNode).open() line: 171
> EnhancedSortMergeJoinStrategy(JoinStrategy).openRight() line: 105
> JoinNode.nextBatchDirect() line: 223
> JoinNode(RelationalNode).nextBatch() line: 282
> GroupingNode$ProjectingTupleSource(BatchCollector$BatchProducerTupleSource).nextTuple() line: 94
> GroupingNode.groupSortPhase() line: 490
> GroupingNode.nextBatchDirect() line: 366
> GroupingNode(RelationalNode).nextBatch() line: 282
> ProjectNode.nextBatchDirect() line: 150
> ProjectNode(RelationalNode).nextBatch() line: 282
> RelationalPlan.nextBatch() line: 145
> QueryProcessor.nextBatchDirect() line: 151
> QueryProcessor.nextBatch() line: 114
> RequestWorkItem$1(BatchCollector).collectTuples(boolean) line: 164
> RequestWorkItem$1(BatchCollector).collectTuples() line: 146
> RequestWorkItem.processMore() line: 472
> RequestWorkItem.process() line: 348
> RequestWorkItem(AbstractWorkItem).run() line: 51
> RequestWorkItem.run() line: 274
> DQPCore.executeRequest(long, RequestMessage, Long) line: 313
> DQPCore.executeRequest(long, RequestMessage) line: 245
> GeneratedMethodAccessor393.invoke(Object, Object[]) line: not available
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> Method.invoke(Object, Object...) line: 606
> LocalServerConnection$1$1.call() line: 177
> FutureTask<V>.run() line: 262
> DQPWorkContext.runInContext(Runnable) line: 276
> DQPWorkContext.runInContext(Callable<V>) line: 260
> LocalServerConnection$1.invoke(Object, Method, Object[]) line: 175
> $Proxy29.executeRequest(long, RequestMessage) line: not available
> StatementImpl.execute(RequestMessage, boolean) line: 673
> StatementImpl.executeSql(String[], boolean, ResultsMode, boolean, RequestOptions, boolean) line: 539
> StatementImpl.execute(String, int) line: 1060
> StatementImpl.execute(String) line: 323
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 9 months