[JBoss JIRA] (TEIID-245) Update costing metadata at runtime
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-245?page=com.atlassian.jira.plugin.... ]
Steven Hawkins updated TEIID-245:
---------------------------------
Priority: Minor (was: Major)
Affects Version/s: (was: 6.0.0)
Lowering the priority. Given the broader ability to pull costing information at import time and the emphasis on smaller more rapid deployments, it isn't as pressing to update the costing at runtime.
> Update costing metadata at runtime
> ----------------------------------
>
> Key: TEIID-245
> URL: https://issues.jboss.org/browse/TEIID-245
> Project: Teiid
> Issue Type: Feature Request
> Components: Connector API, Query Engine
> Reporter: Steven Hawkins
> Priority: Minor
> Fix For: 11.x
>
> Attachments: OracleExecutionFactory.java.patch
>
>
> Defect Tracker #24584: Cost analysis should be both a design-time activity and scheduled/feed during runtime.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 11 months
[JBoss JIRA] (TEIID-147) Support for JAVA Stored Procedures
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-147?page=com.atlassian.jira.plugin.... ]
Steven Hawkins resolved TEIID-147.
----------------------------------
Resolution: Out of Date
Marking as out of date as there's been little movement. I'd like to see progress on TEIID-4932, which coupled with allowing table function UDFs would get us most of the way toward this issue. We can later decide if we want the whole definition capable of being defined directly in the ddl using a language extension.
> Support for JAVA Stored Procedures
> ----------------------------------
>
> Key: TEIID-147
> URL: https://issues.jboss.org/browse/TEIID-147
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Affects Versions: 9.x
> Reporter: Ramesh Reddy
> Fix For: 11.x
>
>
> Currently the Stored procedure support is lacking for both our procedure language and XQuery. Procedure support could be extended to java (or groovy) code, which will provide a powerful engine to execute a arbitrary queries.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 11 months
[JBoss JIRA] (TEIID-4997) Teiid on/with Spark
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4997?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4997:
----------------------------------
Fix Version/s: 11.x
> Teiid on/with Spark
> -------------------
>
> Key: TEIID-4997
> URL: https://issues.jboss.org/browse/TEIID-4997
> Project: Teiid
> Issue Type: Feature Request
> Components: Build/Kits, Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 11.x
>
>
> With the availability of Spark on OpenShift, we should provide a cooperative planning/execution mode for Teiid that utilizes the Spark engine.
> Roughly this would look like a Teiid master running embedded with the Spark master serving the typical JDBC/ODBC/OData endpoints. On an incoming query the optimizer would choose to process against Spark or to process with Teiid - if processing with Teiid that may still require submitting the job to a worker to avoid burdening the master. Alternatively the Teiid master could run in a separate pod with the additional serialization costs, however initially the remote Spark [JDBC/ODBC layer|https://spark.apache.org/docs/latest/sql-programming-guide.html#dis...] will not be available in the OpenShift effort.
> If execution against Spark is chosen, then instead of a typical Teiid processor plan a spark job will be created instead. Initially this could be limited to relational plans, but could be expanded to include procedure language support translated to python, scala, etc. The spark job would represent each source access as a [temporary view|https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc...] accessing the relevant pushdown query. Ideally this would be executed against a Teiid Embedded instance running in the worker node. If remote this would incur an extra hop and have security considerations. This can be thought of as using Teiid for its virtualization and access layer features. The rest of the processing about the access layers could then be represented as Spark SQL.
> For example a Teiid user query of "select * from hdfs.tbl h, oracle.tbl o where h.id = o.id order by h.col" would become the Spark SQL job:
> CREATE TEMPORARY VIEW h
> USING org.apache.spark.sql.jdbc
> OPTIONS (
> url "jdbc:teiid:vdb",
> dbtable "(select col ... from hdfs.tbl)",
> fetchSize '1024,
> ...
> )
> CREATE TEMPORARY VIEW o
> USING org.apache.spark.sql.jdbc
> OPTIONS (
> url "jdbc:teiid:vdb",
> dbtable "(select col ... from oracle.tbl)",
> fetchSize '1024,
> ...
> )
> SELECT * FROM h inner join o on h.id
> The challenges/considerations of this are:
> * Utilizing embedded with coordinated VDB management. There's the associated issue of driver management as well.
> * Translating Teiid SQL to Spark SQL. All Teiid functions, udfs, aggregate functions would need to be made known to Spark. Table function constructs, such as XMLTABLE, TEXTTABLE, etc. could initially just be treated as access layer concerns. Type issues would exist as xml/clob/json would map to string.
> * no xa support
> * we'd need to provide reasonable values for fetch size, partition information, etc. in the access layer queries.
> * We'd have to determine the extent to which federated join optimizations need to be conveyed (dependent join and pushdown) as that would go beyond simply translating to Spark SQL.
> * there's a potential to use [global temporary views|http://www.gatorsmile.io/globaltempview/] which is a more convenient way of adding virtualization to Spark.
> * Large internal materialization should be re-targeted to Spark or JDG
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 11 months
[JBoss JIRA] (TEIID-4612) Infinistpa DSL translator - table names are case sensitive
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4612?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-4612:
------------------------------------
Now the JDG 7.x has been supported and this issue hasn't been raised and JDG 6.x. going to defer this.
> Infinistpa DSL translator - table names are case sensitive
> ----------------------------------------------------------
>
> Key: TEIID-4612
> URL: https://issues.jboss.org/browse/TEIID-4612
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector
> Affects Versions: 8.12.8.6_3
> Reporter: Juraj Duráni
> Assignee: Van Halbert
>
> Table names defined in DDL metadata are case sensitive.
> Example:
> This table name has exact name as object in JDG cache
> {code:sql|title=DDL metadata}
> CREATE FOREIGN TABLE SmallA (intKey integer PRIMARY KEY,...
> {code}
> This query executes correctly:
> {code:sql|title=Query}
> insert into smalla (intkey, stringkey) values (1, '1');
> {code}
> However, changing metadata to this:
> {code:sql|title=Updated DDL metadata}
> CREATE FOREIGN TABLE SMALLA (intKey integer PRIMARY KEY,...
> {code}
> Ends with error:
> {code:plain}
> 14:58:31,509 WARN [org.teiid.CONNECTOR] (Worker17_QueryProcessorQueue224) Connector worker process failed for atomic-request=sUTIOW6woCs0.0.0.57: org.teiid.translator.TranslatorException: TEIID21005 Registered Class not found for table reference SMALLA, check table name or nameInSouce to ensure one matches a registered class in the resource adapter
> at org.teiid.translator.object.util.ObjectUtil.getRegisteredClass(ObjectUtil.java:106) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.executeUpdate(ObjectUpdateExecution.java:121) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.execute(ObjectUpdateExecution.java:100) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem$1.execute(ConnectorWorkItem.java:404) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
> {code}
> The error happens even with specified NAMEINSOURCE for the table:
> {code:sql|title=DDL metadata}
> CREATE FOREIGN TABLE SmallA (intKey integer PRIMARY KEY,...
> OPTIONS (NAMEINSOURCE 'SmallA', ...)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 11 months
[JBoss JIRA] (TEIID-4612) Infinistpa DSL translator - table names are case sensitive
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4612?page=com.atlassian.jira.plugin... ]
Van Halbert closed TEIID-4612.
------------------------------
Resolution: Deferred
> Infinistpa DSL translator - table names are case sensitive
> ----------------------------------------------------------
>
> Key: TEIID-4612
> URL: https://issues.jboss.org/browse/TEIID-4612
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector
> Affects Versions: 8.12.8.6_3
> Reporter: Juraj Duráni
> Assignee: Van Halbert
>
> Table names defined in DDL metadata are case sensitive.
> Example:
> This table name has exact name as object in JDG cache
> {code:sql|title=DDL metadata}
> CREATE FOREIGN TABLE SmallA (intKey integer PRIMARY KEY,...
> {code}
> This query executes correctly:
> {code:sql|title=Query}
> insert into smalla (intkey, stringkey) values (1, '1');
> {code}
> However, changing metadata to this:
> {code:sql|title=Updated DDL metadata}
> CREATE FOREIGN TABLE SMALLA (intKey integer PRIMARY KEY,...
> {code}
> Ends with error:
> {code:plain}
> 14:58:31,509 WARN [org.teiid.CONNECTOR] (Worker17_QueryProcessorQueue224) Connector worker process failed for atomic-request=sUTIOW6woCs0.0.0.57: org.teiid.translator.TranslatorException: TEIID21005 Registered Class not found for table reference SMALLA, check table name or nameInSouce to ensure one matches a registered class in the resource adapter
> at org.teiid.translator.object.util.ObjectUtil.getRegisteredClass(ObjectUtil.java:106) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.executeUpdate(ObjectUpdateExecution.java:121) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.execute(ObjectUpdateExecution.java:100) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem$1.execute(ConnectorWorkItem.java:404) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
> {code}
> The error happens even with specified NAMEINSOURCE for the table:
> {code:sql|title=DDL metadata}
> CREATE FOREIGN TABLE SmallA (intKey integer PRIMARY KEY,...
> OPTIONS (NAMEINSOURCE 'SmallA', ...)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 11 months
[JBoss JIRA] (TEIID-4612) Infinistpa DSL translator - table names are case sensitive
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4612?page=com.atlassian.jira.plugin... ]
Work on TEIID-4612 stopped by Van Halbert.
------------------------------------------
> Infinistpa DSL translator - table names are case sensitive
> ----------------------------------------------------------
>
> Key: TEIID-4612
> URL: https://issues.jboss.org/browse/TEIID-4612
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector
> Affects Versions: 8.12.8.6_3
> Reporter: Juraj Duráni
> Assignee: Van Halbert
>
> Table names defined in DDL metadata are case sensitive.
> Example:
> This table name has exact name as object in JDG cache
> {code:sql|title=DDL metadata}
> CREATE FOREIGN TABLE SmallA (intKey integer PRIMARY KEY,...
> {code}
> This query executes correctly:
> {code:sql|title=Query}
> insert into smalla (intkey, stringkey) values (1, '1');
> {code}
> However, changing metadata to this:
> {code:sql|title=Updated DDL metadata}
> CREATE FOREIGN TABLE SMALLA (intKey integer PRIMARY KEY,...
> {code}
> Ends with error:
> {code:plain}
> 14:58:31,509 WARN [org.teiid.CONNECTOR] (Worker17_QueryProcessorQueue224) Connector worker process failed for atomic-request=sUTIOW6woCs0.0.0.57: org.teiid.translator.TranslatorException: TEIID21005 Registered Class not found for table reference SMALLA, check table name or nameInSouce to ensure one matches a registered class in the resource adapter
> at org.teiid.translator.object.util.ObjectUtil.getRegisteredClass(ObjectUtil.java:106) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.executeUpdate(ObjectUpdateExecution.java:121) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.execute(ObjectUpdateExecution.java:100) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem$1.execute(ConnectorWorkItem.java:404) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) [:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
> {code}
> The error happens even with specified NAMEINSOURCE for the table:
> {code:sql|title=DDL metadata}
> CREATE FOREIGN TABLE SmallA (intKey integer PRIMARY KEY,...
> OPTIONS (NAMEINSOURCE 'SmallA', ...)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
5 years, 11 months