[JBoss JIRA] (TEIID-2381) Expanded source hint support
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-2381:
-------------------------------------
Summary: Expanded source hint support
Key: TEIID-2381
URL: https://issues.jboss.org/browse/TEIID-2381
Project: Teiid
Issue Type: Enhancement
Components: Query Engine
Reporter: Steven Hawkins
Assignee: Steven Hawkins
We currently look at the source hint in only the root user query (not in subqueries nor the with clause) and only consider it in a very narrow set of circumstances when it's used in a view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-2249) Enable the use of temporary tables for those data sources that support them instead of IN criteria for EDS
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2249?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2249:
---------------------------------------
There is current logic that unconditionally allows for the pushdown of dependent joins. In essence the tuplebuffer(s) backing the independent side are made available at the translator level to be dealt with via customization - there is no built-in handling at the translator level.
To provide a solution around temp tables we have to consider:
* when should the dependent join pushdown occur (based upon the raw number of input keys, the potential number of source queries to spawn, etc.)
* what handling is needed at the translator level, assuming the conditional pushdown logic is sufficient then the only handling we would expect is then need to create a temporary table. If the conditional logic is not sufficient then it seems like the translator level would need most of the engine strategy / logic such as splitting across predicates/queries.
* additional concerns not yet specified, such as what schema should be used, the direction of the join (if indexes would need to be added to the temporary table), etc.
> Enable the use of temporary tables for those data sources that support them instead of IN criteria for EDS
> ----------------------------------------------------------------------------------------------------------
>
> Key: TEIID-2249
> URL: https://issues.jboss.org/browse/TEIID-2249
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Debbie Steigner
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> Our proposal is to allow for the more efficient use of large ad-hoc result-sets by rather than creating a long 'IN' list, inserting them in to a temporary table - for example a # table in Sybase and SQL Server - and then generating an SQL join to that instead.
> One of the difference to materialized views (or at least my understanding), is that this work happens at a data-source rather than within the Teiid server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-196) Support creation of temp tables on physical sources.
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-196?page=com.atlassian.jira.plugin.... ]
Steven Hawkins commented on TEIID-196:
--------------------------------------
Added the initial implementation, but did not include the AS clause support - since it is just syntactic sugar for a separate native call. The body of the table definition is exactly the same as what is supported via our DDL metadata. The ON clause references a model/schema name, but is only the logically source for the entry - we do not consider the table as belonging to that schema and it will have the same scoping semantics as a normal temporary table.
Additional work:
* security considerations - we have a allow temp table permission, but that may not be specific enough. Unless we disallow the use of the native-query construct (which is an option), then creating a temporary foreign provides a path for relatively arbitrary sql. Also there's no restriction on what source table may be referenced by the create which also implies the need for more restrictive/specific permissions.
* ease of use as an internal temporary table replacement - the initial check-in assumes that the user takes all responsibility for any native statements required to create/drop the source table (which can either be issued directly against the source or via a Teiid native procedure exec). The sql also relies on DDL, which means it defaults to non-updatable and to be fully useful against a particular source may need to be littered with native-type extension metadata, etc. These issues do not make this an easy replacement for internal temporary tables. However on the flip side of this there are lots of issues with assuming full responsibility with the creation of source temporary tables this includes:
** the need for full bi-directional type mapping (our logic is currently mostly unidirectional)
** sql generation (which in many cases could be lacking given source specific features that aren't easily captured by Teiid)
** source scoping/cleanup - since we will generally be using multiple connections for a given user session, the temp scoping would need to be global and server shutdown or other abnormal behavior would imply that we'll need to clean-up those resources later.
> Support creation of temp tables on physical sources.
> ----------------------------------------------------
>
> Key: TEIID-196
> URL: https://issues.jboss.org/browse/TEIID-196
> Project: Teiid
> Issue Type: Feature Request
> Components: Connector API, Query Engine
> Affects Versions: 6.0.0
> Reporter: Ken Johnson
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> This is a multi-part request.
> First, the system should support creation of temporary tables using a physical backing store rather than buffer manger. Given multi-pass SQL's heavy use of temp tables, buffer manager can easily be overloaded with large interim results stored in temp tables.
> Second, this should be a user-configurable behavior. For example, user might be able to choose a system-level or session-level default from among:
> -- memory/cache
> -- a source represented by a connector binding
> -- a distinct temp source defined with it's own connection parameters (possibly another schema in the repository DB instance)
> Ideally default selectoin should be override-able at temp table creation time through a DDL extension
> In the case where multiple temp tables have been created on a source via connector, the query planner should recognize this and leverage pushdown to the temp store when later query passes access multiple temp tables.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-2349) Cannot execute Two-Phase Commit using Multi-Source Model Dynamic VDB
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-2349?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-2349:
------------------------------------------------
Filip Elias <felias(a)redhat.com> made a comment on [bug 895304|https://bugzilla.redhat.com/show_bug.cgi?id=895304]
Verified on SOA-P 5.3.1 ER4
> Cannot execute Two-Phase Commit using Multi-Source Model Dynamic VDB
> --------------------------------------------------------------------
>
> Key: TEIID-2349
> URL: https://issues.jboss.org/browse/TEIID-2349
> Project: Teiid
> Issue Type: Bug
> Components: Server
> Affects Versions: 7.7.1
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 8.1, 7.7.4
>
>
> Description of problem:
> 2nd connection to Multi-Source Model Dynamic VDB is not Two-Phase Commit.
> How reproducible:
> Always
> Steps to Reproduce:
> Execute a update query via Multi-Source Model Dynamic VDB.
> ex)
> vdb - maseter-vdb.xml
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <vdb name="MASTER-VDB" version="1">
> <description>A Dynamic VDB</description>
> <property name="UseConnectorMetadata" value="true" />
> <model visible="true" type="PHYSICAL" name="VDB1">
> <property name="supports-multi-source-bindings" value="true" />
> <source name="db02" translator-name="postgresql" connection-jndi-name="java:/PostgresXADS02" />
> <source name="db01" translator-name="postgresql" connection-jndi-name="java:/PostgresXADS01" />
> </model>
> </vdb>
>
> Additional info:
> Set the multisourceUpdate flag as first query plan creating.
> org.teiid.dqp.internal.process.multisource.MultiSourcePlanToProcessConverter.java
> 90: @Override
> 91: public synchronized RelationalPlan convert(PlanNode planNode)
> 92: throws QueryPlannerException, TeiidComponentException {
> 93: RelationalPlan result = null;
> 94: try {
> 95: result = super.convert(planNode);
> 96: return result;
> 97: } finally {
> 98: if (result != null && update && multiSource) {
> -> 99: result.setMultisourceUpdate(true);
> 100: }
> 101: update = false;
> 102: multiSource = false;
> 103: }
> 104: }
> If multisourceUpdate is true, transactionService is started by teiid engine. But, second connects does not set the multisourceUpdate flag. Therefore, teiid engine does not start transactionService.
> org.teiid.dqp.internal.process.Request.java
> 338: private void createProcessor() throws TeiidComponentException {
> --- snip ---
> 349: if(RequestMessage.TXN_WRAP_ON.equals(requestMsg.getTxnAutoWrapMode())){
> 350: startAutoWrapTxn = true;
> 351: } else if (RequestMessage.TXN_WRAP_DETECT.equals(requestMsg.getTxnAutoWrapMode())){
> 352: boolean transactionalRead = requestMsg.getTransactionIsolation() == Connection.TRANSACTION_REPEATABLE_READ
> 353: || requestMsg.getTransactionIsolation() == Connection.TRANSACTION_SERIALIZABLE;
> -> 354: startAutoWrapTxn = processPlan.requiresTransaction(transactionalRead);
> 355: }
> 356:
> 357: if (startAutoWrapTxn) {
> 358: try {
> -> 359: transactionService.begin(tc);
> 360: } catch (XATransactionException err) {
> 361: throw new TeiidComponentException(err);
> 362: }
> 363: }
> 364: }
> org.teiid.query.processor.relational.RelationalPlan.java
> 254: @Override
> 255: public boolean requiresTransaction(boolean transactionalReads) {
> 256: if (multisourceUpdate) {
> -> 257: return true;
> 258: }
> 259: if (this.with != null) {
> 260: if (transactionalReads) {
> 261: return true;
> 262: }
> 263: for (WithQueryCommand withCommand : this.with) {
> 264: if (withCommand.getCommand().getProcessorPlan().requiresTransaction(transactionalReads)) {
> 265: return true;
> 266: }
> 267: }
> 268: }
> 279: return requiresTransaction(transactionalReads, root);
> 270: }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-2349) Cannot execute Two-Phase Commit using Multi-Source Model Dynamic VDB
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-2349?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-2349:
------------------------------------------------
Filip Elias <felias(a)redhat.com> changed the Status of [bug 895304|https://bugzilla.redhat.com/show_bug.cgi?id=895304] from ON_QA to VERIFIED
> Cannot execute Two-Phase Commit using Multi-Source Model Dynamic VDB
> --------------------------------------------------------------------
>
> Key: TEIID-2349
> URL: https://issues.jboss.org/browse/TEIID-2349
> Project: Teiid
> Issue Type: Bug
> Components: Server
> Affects Versions: 7.7.1
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 8.1, 7.7.4
>
>
> Description of problem:
> 2nd connection to Multi-Source Model Dynamic VDB is not Two-Phase Commit.
> How reproducible:
> Always
> Steps to Reproduce:
> Execute a update query via Multi-Source Model Dynamic VDB.
> ex)
> vdb - maseter-vdb.xml
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <vdb name="MASTER-VDB" version="1">
> <description>A Dynamic VDB</description>
> <property name="UseConnectorMetadata" value="true" />
> <model visible="true" type="PHYSICAL" name="VDB1">
> <property name="supports-multi-source-bindings" value="true" />
> <source name="db02" translator-name="postgresql" connection-jndi-name="java:/PostgresXADS02" />
> <source name="db01" translator-name="postgresql" connection-jndi-name="java:/PostgresXADS01" />
> </model>
> </vdb>
>
> Additional info:
> Set the multisourceUpdate flag as first query plan creating.
> org.teiid.dqp.internal.process.multisource.MultiSourcePlanToProcessConverter.java
> 90: @Override
> 91: public synchronized RelationalPlan convert(PlanNode planNode)
> 92: throws QueryPlannerException, TeiidComponentException {
> 93: RelationalPlan result = null;
> 94: try {
> 95: result = super.convert(planNode);
> 96: return result;
> 97: } finally {
> 98: if (result != null && update && multiSource) {
> -> 99: result.setMultisourceUpdate(true);
> 100: }
> 101: update = false;
> 102: multiSource = false;
> 103: }
> 104: }
> If multisourceUpdate is true, transactionService is started by teiid engine. But, second connects does not set the multisourceUpdate flag. Therefore, teiid engine does not start transactionService.
> org.teiid.dqp.internal.process.Request.java
> 338: private void createProcessor() throws TeiidComponentException {
> --- snip ---
> 349: if(RequestMessage.TXN_WRAP_ON.equals(requestMsg.getTxnAutoWrapMode())){
> 350: startAutoWrapTxn = true;
> 351: } else if (RequestMessage.TXN_WRAP_DETECT.equals(requestMsg.getTxnAutoWrapMode())){
> 352: boolean transactionalRead = requestMsg.getTransactionIsolation() == Connection.TRANSACTION_REPEATABLE_READ
> 353: || requestMsg.getTransactionIsolation() == Connection.TRANSACTION_SERIALIZABLE;
> -> 354: startAutoWrapTxn = processPlan.requiresTransaction(transactionalRead);
> 355: }
> 356:
> 357: if (startAutoWrapTxn) {
> 358: try {
> -> 359: transactionService.begin(tc);
> 360: } catch (XATransactionException err) {
> 361: throw new TeiidComponentException(err);
> 362: }
> 363: }
> 364: }
> org.teiid.query.processor.relational.RelationalPlan.java
> 254: @Override
> 255: public boolean requiresTransaction(boolean transactionalReads) {
> 256: if (multisourceUpdate) {
> -> 257: return true;
> 258: }
> 259: if (this.with != null) {
> 260: if (transactionalReads) {
> 261: return true;
> 262: }
> 263: for (WithQueryCommand withCommand : this.with) {
> 264: if (withCommand.getCommand().getProcessorPlan().requiresTransaction(transactionalReads)) {
> 265: return true;
> 266: }
> 267: }
> 268: }
> 279: return requiresTransaction(transactionalReads, root);
> 270: }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-2380) Planning failure with update subquery compensation with agg comparision
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2380?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-2380.
-----------------------------------
Resolution: Done
Updated the rewrite to use the same symbols from the select in the order by, just as the normal resolving path does.
> Planning failure with update subquery compensation with agg comparision
> -----------------------------------------------------------------------
>
> Key: TEIID-2380
> URL: https://issues.jboss.org/browse/TEIID-2380
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 7.3
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> An update of the form:
> delete from pm3.g1 where e1 = (SELECT MAX(e1) FROM pm3.g1 as z where e2 = pm3.g1.e2)
> Such that the subquery cannot be pushed will result in the unaliased max being known by two different names which results in an AssertionError in the SortNode initialization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-2380) Planning failure with update subquery compensation with agg comparision
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-2380:
-------------------------------------
Summary: Planning failure with update subquery compensation with agg comparision
Key: TEIID-2380
URL: https://issues.jboss.org/browse/TEIID-2380
Project: Teiid
Issue Type: Bug
Components: Query Engine
Affects Versions: 7.3
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 8.3
An update of the form:
delete from pm3.g1 where e1 = (SELECT MAX(e1) FROM pm3.g1 as z where e2 = pm3.g1.e2)
Such that the subquery cannot be pushed will result in the unaliased max being known by two different names which results in an AssertionError in the SortNode initialization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (TEIID-2379) NPE with uninitialized sub plan on close
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-2379:
-------------------------------------
Summary: NPE with uninitialized sub plan on close
Key: TEIID-2379
URL: https://issues.jboss.org/browse/TEIID-2379
Project: Teiid
Issue Type: Bug
Components: Query Engine
Affects Versions: 7.5
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 8.3
If a PlanExecutionNode is not opened, the contained plan is not initialized, but we still attempt to close it which can throw an NPE on a null BufferManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months