[JBoss JIRA] (TEIID-4579) External Materialization MySQL can't swap primary and stage table in xa transaction
by Jan Stastny (JIRA)
Jan Stastny created TEIID-4579:
----------------------------------
Summary: External Materialization MySQL can't swap primary and stage table in xa transaction
Key: TEIID-4579
URL: https://issues.jboss.org/browse/TEIID-4579
Project: Teiid
Issue Type: Bug
Components: Server
Affects Versions: 8.12.7.6_3
Reporter: Jan Stastny
Assignee: Steven Hawkins
When using MySQL 5.5 as materialization target (primary and stage table are placed there) and on server side the mysql datasource is configured as xa-datasource, problem with transaction scope and status appears.
The cause of this issue is, that in my "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" I try to swap primary and stage materialization tables by using third swap table:
{code:sql}
exec Source.native(''ALTER TABLE DV_MATVIEWS_MAT_VIEW_STAGE RENAME TO DV_MATVIEWS_MAT_VIEW_TEMP'');
exec Source.native(''ALTER TABLE DV_MATVIEWS_MAT_VIEW RENAME TO DV_MATVIEWS_MAT_VIEW_STAGE'');
exec Source.native(''ALTER TABLE DV_MATVIEWS_MAT_VIEW_TEMP RENAME TO DV_MATVIEWS_MAT_VIEW'');
{code}
but there is a limitation of ALTER TABLE commands which results in implicit commit:
https://dev.mysql.com/doc/refman/5.5/en/implicit-commit.html
Eventually the first of the three ALTER commands succeeds, but the second fails, thus I end up with no DV_MATVIEWS_MAT_VIEW_STAGE table, as it was previously renamed to DV_MATVIEWS_MAT_VIEW_TEMP. This particular problem could be resolved by TEIID-4240 I believe.
The stacktrace of the issue:
{code:plain}
15:19:59,825 DEBUG [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) dEAkWiMWXtw2.0.77.13 Processing NEW request: EXEC Source.native('ALTER TABLE DV_MATVIEWS_MAT_VIEW_STAGE RENAME TO DV_MATVIEWS_MAT_VIEW_TEMP')
15:20:00,035 DEBUG [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) dEAkWiMWXtw2.0.77.13 Obtained execution
15:20:00,036 DEBUG [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) Source-specific command: ALTER TABLE DV_MATVIEWS_MAT_VIEW_STAGE RENAME TO DV_MATVIEWS_MAT_VIEW_TEMP
15:20:00,247 WARN [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) Connector worker process failed for atomic-request=dEAkWiMWXtw2.0.77.13: org.teiid.translator.jdbc.JDBCExecutionException: 1399 TEIID11008:TEIID11004 Error executing statement(s): ALTER TABLE DV_MATVIEWS_MAT_VIEW_STAGE RENAME TO DV_MATVIEWS_MAT_VIEW_TEMP
at org.teiid.translator.jdbc.JDBCDirectQueryExecution.execute(JDBCDirectQueryExecution.java:95) [translator-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1]
at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_102]
at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
at com.sun.proxy.$Proxy81.execute(Unknown Source)
at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:142)
at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:391)
at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:150)
at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:145)
at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)
at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:69)
at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:70)
at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:84)
at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:92)
at org.teiid.query.processor.proc.ProcedurePlan.executePlan(ProcedurePlan.java:608)
at org.teiid.query.processor.proc.CreateCursorResultSetInstruction.process(CreateCursorResultSetInstruction.java:69)
at org.teiid.query.processor.proc.ExecDynamicSqlInstruction$1.process(ExecDynamicSqlInstruction.java:218)
at org.teiid.query.processor.proc.ProcedurePlan.processProcedure(ProcedurePlan.java:389)
at org.teiid.query.processor.proc.ProcedurePlan.nextBatchDirect(ProcedurePlan.java:298)
at org.teiid.query.processor.proc.ProcedurePlan.nextBatch(ProcedurePlan.java:270)
at org.teiid.query.processor.relational.PlanExecutionNode.nextBatchDirect(PlanExecutionNode.java:118)
at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:150)
at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:145)
at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)
at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:164)
at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146)
at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:472)
at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:348)
at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)
at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274)
at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_102]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_102]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_102]
Caused by: java.sql.SQLException: XAER_RMFAIL: The command cannot be executed when global transaction is in the ACTIVE state
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2713)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2663)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:888)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:730)
at com.mysql.jdbc.jdbc2.optional.StatementWrapper.execute(StatementWrapper.java:698)
at org.jboss.jca.adapters.jdbc.WrappedStatement.execute(WrappedStatement.java:157)
at org.teiid.translator.jdbc.JDBCDirectQueryExecution.execute(JDBCDirectQueryExecution.java:83) [translator-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1]
... 45 more
15:20:00,254 DEBUG [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) dEAkWiMWXtw2.0.77.13 Remove State
15:20:00,254 DEBUG [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) dEAkWiMWXtw2.0.77.13 Processing Close : EXEC Source.native('ALTER TABLE DV_MATVIEWS_MAT_VIEW_STAGE RENAME TO DV_MATVIEWS_MAT_VIEW_TEMP')
{code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (TEIID-4573) Modeshape query
by Horia Chiorean (JIRA)
[ https://issues.jboss.org/browse/TEIID-4573?page=com.atlassian.jira.plugin... ]
Horia Chiorean commented on TEIID-4573:
---------------------------------------
[~van.halbert]: you are correct, the grammar for {{ComparisonConstraint}} looks like:
{code}
Comparison ::= DynamicOperand Operator StaticOperand
{code}
so in the previous query the right hand side of the {{=}} constraint would not work anyway.
I was referring to the fact that {{DynamicOperand}} for {{PropertyValue}} looks like this:
{code}
PropertyValue ::= [selectorName'.'] propertyName
{code}
and {{propertyName}} conforms to the "generic" Name grammar which is
{code}
Name ::= '[' quotedName ']' | '[' simpleName ']' | simpleName
{code}
So in the above query, on the left-hand side (assuming the right-hand side was a {{StaticOperand}}), using {{selector."propertyName"}} would not be a valid construct.
> Modeshape query
> ---------------
>
> Key: TEIID-4573
> URL: https://issues.jboss.org/browse/TEIID-4573
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.0
> Reporter: Tom Johnston
> Assignee: Steven Hawkins
> Labels: modeshape
> Fix For: 9.2, 9.1.2
>
>
> The following modeshape vdb query fails.
> select a.jcr_path, a.table_name ,b.jcr_name
> from "Relational_Model_View"."Table_Columns" a, "ModeShape"."xmi_model" b
> where a.jcr_path = b.jcr_path
> the error is:
> Error: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 ModeShape: 0 TEIID11008:TEIID11004 Error executing statement(s): [SQL: SELECT g_1."jcr:path", g_0."mode:localName", g_2."jcr:name" FROM "relational:table" AS g_0 INNER JOIN "relational:column" AS g_1 ON ISCHILDNODE(g_1, g_0) INNER JOIN "xmi:model" AS g_2 ON g_1."jcr:path" = g_2."jcr:path" AND ISCHILDNODE(g_2, g_0)]
> SQLState: 50000
> ErrorCode: 30504
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (TEIID-4437) Convert Teiid build to use "feature-pack"
by Kylin Soong (JIRA)
[ https://issues.jboss.org/browse/TEIID-4437?page=com.atlassian.jira.plugin... ]
Kylin Soong resolved TEIID-4437.
--------------------------------
Resolution: Done
Due to the following 2 aspects reason set this issue to resolved:
* All related(including the sub-jira) pull request be merged
* Junit tests, Arquillian tests, Test with quick start examples all works fine
> Convert Teiid build to use "feature-pack"
> -----------------------------------------
>
> Key: TEIID-4437
> URL: https://issues.jboss.org/browse/TEIID-4437
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits
> Reporter: Ramesh Reddy
> Assignee: Kylin Soong
> Fix For: 9.2
>
>
> Change the current build process to build "feature-packs" for project, and then based on feature pack, build
> - WildFly Server distribution
> -WildFly Server Overlay distribution
> Teiid currently have distributions, but not based on feature-packs, they need to be. Other distributions like AdminShell do not fall into this category. Only distributions that has WF involvement need a feature -pack (this is important to remember).
> Currently Teiid uses "kit" directory in many different projects along with "wildfly-dist.xml" to populate "modules" directory. A feature-pack does EXACTLY same thing, but more with checks and balances. A feature pack, makes sure all the dependencies are met based on module.xml and module.xml defines the maven artifact rather than JAR file. i.e. it defines a metadata file, from which it can be build the zip file that "wildfly-dist.xml" assembly is currently doing. It also can has capabilities to "configure" the WF subsystem. That is manually done or done through CLI currently, using this part will be new.
> Implementation Rules
> 1) Every where we have "kit" directory, that module REQUIRES a "feature-pack" module. That means, every translator and resource-adapter gets it's own, and Teiid engine modules separately gets its own module. The reason to keep/create multiple modules is STRICTLY to keep the "modules" directory in feature-pack aligned ONE TO ONE with project that feature-pack represents. Otherwise, as developer we will NOT know which project is bringing in certain dependency, if all the modules are in single place. So, this is more about management of dependencies in a sane way. (Teiid did have all the modules in once place before in 7.x/8.x time, it is a nightmare to handle growing dependencies with our translators)
> 2) NONE of the work we have is useless/or need to be redone, most everything we have PERFECTLY maps to this effort, except for for configuration part.
> 3) For translator and resource adapter pair, I recommend that we pull them into single sub folder, and create a single feature pack for it. For example:
> we have
> {code}
> connectors/translator-mongodb
> connectors/connector-mongodb
> {code}
> I would like see we design as
> {code}
> connectors
> /mongdb
> /translator-mongodb
> /connector-mongodb
> /feature-pack-mongodb
> {code}
> No need to create "dist" packages for any resource-adapters or translators. As I mentioned in the beginning of the issue, there are two distributions, they are built using these various feature-packs.
> 4) I would like to see a clean move of "kit/wildfly/modules" to "feature-packs" using "git", NOT CUT n PASTE where we loose the GIT history. This will preserve the confidence that what we have currently is EXACTLY same as in future with feature-packs.
> 5) When designing the "feature-pack" projects, they ABSOLUTELY MUST NOT define any dependencies in their pom.xml other than Teiid project(s) they represent and any other feature-pack they refer to. (Kylin this is what cause most concern from your design for me)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (TEIID-4565) Add objects table to system schema
by Tom Johnston (JIRA)
[ https://issues.jboss.org/browse/TEIID-4565?page=com.atlassian.jira.plugin... ]
Tom Johnston edited comment on TEIID-4565 at 11/14/16 2:23 PM:
---------------------------------------------------------------
I would see that the objects system table would likely NOT include properties but their associated properties would be joinable to this new table... similar to my original use case query.
Searching for objects by name (and/or/ their potential descriptions) is the driving force.
was (Author: virtualdatabase):
I would see that the objects system table would like include properties but their associated properties would be joinable to this new table... similar to my original use case query.
Searching for objects by name (and/or/ their potential descriptions) is the driving force.
> Add objects table to system schema
> ----------------------------------
>
> Key: TEIID-4565
> URL: https://issues.jboss.org/browse/TEIID-4565
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Tom Johnston
> Assignee: Steven Hawkins
>
> The addition of a objects table would make it simpler to join to when extensions are implemented.
> An example might be something similar to this query:
> select sysObjects.*
> from (
> select t.schemaName as Parent,t.name, t.uid ,'Table' as ObjectType
> from sys.tables t
> where schemaName not in (select distinct(schemaName) from "SYS"."Tables" where isSystem = true)
> UNION
> select c.tableName as Parent,c.name, c.uid,'Column' as ObjectType
> from sys.columns c
> where schemaName not in (select distinct(schemaName) from "SYS"."Tables" where isSystem = true)
> UNION
> select null as Parent,e.Value as Name,e.uid,'Property' as ObjectType
> from sys.properties as e) as SysObjects
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (TEIID-4565) Add objects table to system schema
by Tom Johnston (JIRA)
[ https://issues.jboss.org/browse/TEIID-4565?page=com.atlassian.jira.plugin... ]
Tom Johnston commented on TEIID-4565:
-------------------------------------
I would see that the objects system table would like include properties but their associated properties would be joinable to this new table... similar to my original use case query.
Searching for objects by name (and/or/ their potential descriptions) is the driving force.
> Add objects table to system schema
> ----------------------------------
>
> Key: TEIID-4565
> URL: https://issues.jboss.org/browse/TEIID-4565
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Tom Johnston
> Assignee: Steven Hawkins
>
> The addition of a objects table would make it simpler to join to when extensions are implemented.
> An example might be something similar to this query:
> select sysObjects.*
> from (
> select t.schemaName as Parent,t.name, t.uid ,'Table' as ObjectType
> from sys.tables t
> where schemaName not in (select distinct(schemaName) from "SYS"."Tables" where isSystem = true)
> UNION
> select c.tableName as Parent,c.name, c.uid,'Column' as ObjectType
> from sys.columns c
> where schemaName not in (select distinct(schemaName) from "SYS"."Tables" where isSystem = true)
> UNION
> select null as Parent,e.Value as Name,e.uid,'Property' as ObjectType
> from sys.properties as e) as SysObjects
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (TEIID-4573) Modeshape query
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4573?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4573:
---------------------------------------
[~hchiorean] from the grammar it seems to specify that you can only do a dynamic to static comparison such that the right hand side is expected to be literal. Aren't the double quotes here just identifier quoting - how does that affect how the reference is interpreted?
> Modeshape query
> ---------------
>
> Key: TEIID-4573
> URL: https://issues.jboss.org/browse/TEIID-4573
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.0
> Reporter: Tom Johnston
> Assignee: Steven Hawkins
> Labels: modeshape
> Fix For: 9.2, 9.1.2
>
>
> The following modeshape vdb query fails.
> select a.jcr_path, a.table_name ,b.jcr_name
> from "Relational_Model_View"."Table_Columns" a, "ModeShape"."xmi_model" b
> where a.jcr_path = b.jcr_path
> the error is:
> Error: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 ModeShape: 0 TEIID11008:TEIID11004 Error executing statement(s): [SQL: SELECT g_1."jcr:path", g_0."mode:localName", g_2."jcr:name" FROM "relational:table" AS g_0 INNER JOIN "relational:column" AS g_1 ON ISCHILDNODE(g_1, g_0) INNER JOIN "xmi:model" AS g_2 ON g_1."jcr:path" = g_2."jcr:path" AND ISCHILDNODE(g_2, g_0)]
> SQLState: 50000
> ErrorCode: 30504
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months
[JBoss JIRA] (TEIID-4573) Modeshape query
by Horia Chiorean (JIRA)
[ https://issues.jboss.org/browse/TEIID-4573?page=com.atlassian.jira.plugin... ]
Horia Chiorean edited comment on TEIID-4573 at 11/14/16 1:11 PM:
-----------------------------------------------------------------
[~van.halbert] the WHERE clause should not use {{"}} quotes when testing for JCR properties. It should either use {{\[jcr:path\]}} (square brackets) or simply nothing {{g_1.jcr:path = g_2.jcr:path}}
was (Author: hchiorean):
[~van.halbert] the WHERE clause should not use {{"}} quotes when testing for JCR properties. It should either use {{\[jcr:path\]}} or simply nothing {{g_1.jcr:path = g_2.jcr:path}}
> Modeshape query
> ---------------
>
> Key: TEIID-4573
> URL: https://issues.jboss.org/browse/TEIID-4573
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.0
> Reporter: Tom Johnston
> Assignee: Steven Hawkins
> Labels: modeshape
> Fix For: 9.2, 9.1.2
>
>
> The following modeshape vdb query fails.
> select a.jcr_path, a.table_name ,b.jcr_name
> from "Relational_Model_View"."Table_Columns" a, "ModeShape"."xmi_model" b
> where a.jcr_path = b.jcr_path
> the error is:
> Error: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 ModeShape: 0 TEIID11008:TEIID11004 Error executing statement(s): [SQL: SELECT g_1."jcr:path", g_0."mode:localName", g_2."jcr:name" FROM "relational:table" AS g_0 INNER JOIN "relational:column" AS g_1 ON ISCHILDNODE(g_1, g_0) INNER JOIN "xmi:model" AS g_2 ON g_1."jcr:path" = g_2."jcr:path" AND ISCHILDNODE(g_2, g_0)]
> SQLState: 50000
> ErrorCode: 30504
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 4 months