[JBoss JIRA] (TEIID-5784) OData4 returns $metadata in JSON when "Accept" not specified
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5784?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5784.
-----------------------------------
Fix Version/s: (was: 12.3)
Resolution: Rejected
This is working in both the odata unit and integration test - which deploys a vdb using the odata4 translator hitting one of our vdb odata endpoints.
> OData4 returns $metadata in JSON when "Accept" not specified
> ------------------------------------------------------------
>
> Key: TEIID-5784
> URL: https://issues.jboss.org/browse/TEIID-5784
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors, OData
> Affects Versions: 12.2
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Priority: Major
>
> with upgrade to Olingo 4.6, there is update to $metadata call that can return the content in JSON format. However OData4 translator can only parse the XML content. This occurs when "Accept" header is not specified with the request
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 4 months
[JBoss JIRA] (TEIID-5436) Materialization translator infinispan-hotrod inconsistent state
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5436?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-5436:
----------------------------------
Fix Version/s: 13.0
(was: 12.3)
> Materialization translator infinispan-hotrod inconsistent state
> ---------------------------------------------------------------
>
> Key: TEIID-5436
> URL: https://issues.jboss.org/browse/TEIID-5436
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.12.6_4
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Critical
> Fix For: 13.0
>
>
> There is an inconsistency between contents of status table and jdg cache when using infinispan-hotrod translator for materialization.
> For a vdb:
> {code:xml}
> <vdb name="teiid5259" version="1">
> <model name="source">
> <source connection-jndi-name="java:/jdg71HotrodDS" name="source"
> translator-name="tr-override" />
> <metadata type="DDL">
> <![CDATA[
> CREATE FOREIGN TABLE a(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${primary.cache.name}');
> CREATE FOREIGN TABLE a_stage(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${staging.cache.name}');
> ]]>
> </metadata>
> <metadata type="NATIVE" />
> </model>
> <model name="h2Source">
> <source name="h2Source" translator-name="h2" connection-jndi-name="java:/h2-localhost-9292-ds" />
> </model>
> <model name="View" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW mat_a(
> id integer NOT NULL
> ) OPTIONS (MATERIALIZED 'TRUE',
> MATERIALIZED_TABLE 'source.a',
> "teiid_rel:MATERIALIZED_STAGE_TABLE" 'source.a_stage',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'execute source.native(''rename source.a_stage source.a'');',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute source.native(''truncate source.a_stage'');',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'THROW_EXCEPTION',
> "teiid_rel:MATVIEW_TTL" 10000
> )
> AS SELECT 1 UNION ALL SELECT 2;
> ]]>
> </metadata>
> </model>
> <translator name="tr-override" type="infinispan-hotrod">
> <property name="SupportsDirectQueryProcedure" value="true"/>
> </translator>
> </vdb>
> {code}
> I can reach a state when:
> # The status table states LOADED for the view
> # The DATA SRC commands inserting into JDG ended (according to teiid-command log)
> # But consecutive query against the view returns lower count (even though having THROW_EXCEPTION in ONERROR_ACTION property)
> See the attached command log:
> {code}
> 13:59:31,679 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) L8MeUp92Tess START USER COMMAND: startTime=2018-03-01 13:59:31.679 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess applicationName=internal principal=embedded-async vdbName=teiid5259 vdbVersion=1 sql=execute SYSADMIN.loadMatView('View','mat_a')
> 13:59:31,680 INFO [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue14) SBqGYWW07OB5 END USER COMMAND: endTime=2018-03-01 13:59:31.68 requestID=SBqGYWW07OB5.0 txID=null sessionID=SBqGYWW07OB5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=SELECT g_0.NAME, g_0.TARGETSCHEMANAME, g_0.TARGETNAME, g_0.VALID, g_0.LOADSTATE, g_0.UPDATED, g_0.CARDINALITY, g_0.LOADNUMBER FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[SELECT g_0."NAME", g_0."TARGETSCHEMANAME", g_0."TARGETNAME", g_0."VALID", g_0."LOADSTATE", g_0."UPDATED", g_0."CARDINALITY", g_0."LOADNUMBER" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:31,716 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue17) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.716 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1071137
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADING', valid = FALSE, Updated = {ts'2018-03-01 13:59:31.719'}, Cardinality = -1 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADING', "VALID" = 0, "UPDATED" = TIMESTAMP '2018-03-01 13:59:31.719', "CARDINALITY" = -1 WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:31,728 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.728 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1169356
> 13:59:31,731 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.731 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('truncate source.a_stage')
> 13:59:31,845 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:31.845 requestID=uaySHcXyc0o5.0 sourceCommandID=74 executionID=6 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=3558736
> 13:59:31,847 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.847 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:32,040 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.04 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=1493046
> 13:59:32,047 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.047 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (1)
> 13:59:32,460 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.46 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4783162
> 13:59:32,462 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.462 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (2)
> 13:59:32,753 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.753 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=0 cpuTime(ns)=5021072
> 13:59:32,800 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.8 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 1, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:32.756'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:32,801 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 SOURCE SRC COMMAND: endTime=2018-03-01 13:59:32.801 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 1, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:32.756', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:32,804 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.804 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=31050942
> 13:59:32,806 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END USER COMMAND: endTime=2018-03-01 13:59:32.806 requestID=uaySHcXyc0o5.0 txID=null sessionID=uaySHcXyc0o5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:33,005 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.005 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4943881
> 13:59:33,014 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.014 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:33,723 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.723 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=4426814
> 13:59:33,731 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.731 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:33.725'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 2)
> 13:59:33,732 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:33.732 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:33.725', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 2]
> 13:59:33,737 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.737 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=3747146
> 13:59:33,740 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END USER COMMAND: endTime=2018-03-01 13:59:33.74 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:35,871 INFO [org.teiid.COMMAND_LOG] (New I/O worker #2) eP8bDNkHEpSG START USER COMMAND: startTime=2018-03-01 13:59:35.871 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG applicationName=JDBC principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 sql=SELECT COUNT(*) FROM mat_a
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.VALID, g_0.LOADSTATE FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG SOURCE SRC COMMAND: endTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sourceCommand=[SELECT g_0."VALID", g_0."LOADSTATE" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:35,878 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue23) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:35.878 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=904441
> 13:59:35,879 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue24) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.879 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.id FROM source.a AS g_0
> 13:59:36,326 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue25) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:36.326 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=32679662
> 13:59:36,384 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue26) eP8bDNkHEpSG END USER COMMAND: endTime=2018-03-01 13:59:36.384 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 4 months
[JBoss JIRA] (TEIID-5801) Communications link failure during commit() error message when copying a MySQL table to a DB
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5801?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5801.
-----------------------------------
Fix Version/s: 12.3
Resolution: Done
Allowing read-only commands to execute in parallel even under a transaction based upon the transaction isolation level.
> Communications link failure during commit() error message when copying a MySQL table to a DB
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-5801
> URL: https://issues.jboss.org/browse/TEIID-5801
> Project: Teiid
> Issue Type: Quality Risk
> Components: Query Engine
> Affects Versions: 12.0
> Environment: teiid-12.0.0 on WildFly Full 14.0.1.Final (WildFly Core 6.0.2.Final)
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.3
>
> Attachments: server_1_fail.log, server_2_works.log, server_teiid.log
>
>
> When copying a MySQL table, for example, to PostgreSQL:
> {code:sql}
> insert into dwh_pg.test_target SELECT * FROM my.test_source ;;
> {code}
> in the end of the process Teiid throws out the following stacktrace (though rows are inserted in PostgreSQL successfully, seems that Teiid can't close read transaction for MySQL):
> {code}
> 2019-08-01 16:48:23,119 WARN [org.jboss.jca.core.connectionmanager.listener.TxConnectionListener] (Worker3_QueryProcessorQueue34) TidBkmeGWJN8 IJ000305: Connection error occured: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@75284e6d[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@5b8d92c7 connection handles=0 lastReturned=1564670796599 lastValidated=1564670796598 lastCheckedOut=1564670796678 trackByTx=true pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@ae21718 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@338041b4[pool=lingoda_read_replica] xaResource=LocalXAResourceImpl@1fcd6b81[connectionListener=75284e6d connectionManager=20f22ec1 warned=false currentXid=null productName=MySQL productVersion=5.6.34-log jndiName=java:/lingoda_read_replica] txSync=TransactionSynchronization@1367866468{tx=Local transaction (delegate=TransactionImple < ac, BasicAction: 0:ffffc0a8008c:33252ff9:5d42fad3:11 status: ActionStatus.PREPARING >, owner=Local transaction context for provider JBoss JTA transaction provider) wasTrackByTx=true enlisted=true cancel=false}]: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1014) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:988) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:974) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919) at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1700) at org.jboss.jca.adapters.jdbc.local.LocalManagedConnection.commit(LocalManagedConnection.java:96) at org.jboss.jca.core.tx.jbossts.LocalXAResourceImpl.commit(LocalXAResourceImpl.java:172) at com.arjuna.ats.internal.jta.resources.arjunacore.XAOnePhaseResource.commit(XAOnePhaseResource.java:120) at com.arjuna.ats.internal.arjuna.abstractrecords.LastResourceRecord.topLevelPrepare(LastResourceRecord.java:152) at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2664) at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2614) at com.arjuna.ats.arjuna.coordinator.BasicAction.prepare(BasicAction.java:2157) at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1503) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1288) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:77) at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71) at org.teiid.dqp.internal.process.TransactionServerImpl.commitDirect(TransactionServerImpl.java:384) at org.teiid.dqp.internal.process.TransactionServerImpl.commit(TransactionServerImpl.java:515) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.teiid.logging.LogManager$LoggingProxy.invoke(LogManager.java:117) at com.sun.proxy.$Proxy25.commit(Unknown Source) at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:514) at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362) at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43) at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285) at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281) at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:113)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:199) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
> {code}
> I tried to reproduce the problem with local MySQL and PostgreSQL but couldn't. The problem can be reproduced only when using remote MySQL and PostgreSQL. On my local machine the error appears with limit more than 200000 rows, on another machines the exception appeared when setting limit 400000 and more. It seems it's related maybe somehow with MySQL timeouts or network delays. If I copy the table from remote MySQL to local PostgeSQL the error doesn't appear, and vice versa, if I copy the table from local MySQL to remote PostgreSQL the error doesn't appear again. I don't have an access to the remote MySQL to have a look at its internal options. I also tried to set net_write_timeout=1800 jdbc property for data source of the remote MySQL, tcpKeepAlive=true, tried to set ThreadBound MySQL translator property to true value - it didn't help at all. What do you think, is it possible to avoid the error on Teiid level?
> I also attached a server log with org.teiid.CONNECTOR and org.teiid.PROCESSOR log outputs.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 4 months
[JBoss JIRA] (TEIID-5802) Row based security and column masking don't work correctly for materialized tables (views)
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5802?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5802:
---------------------------------------
No, I wouldn't add user scoped materializations back as that just introduces a different set of problems. The most conceptually simple and consistent approach is to by pass our security checks when building a materialized view target table and stipulate in the docs that materialization target tables act as their own top level entities for the purposes of column/row security. But obviously that would be a big change for those used to the current behavior to the materialization being populated with security applied based upon whomever triggers the load, so it would need some kind of a flag.
We are not alone in this problem, for example with pg: https://www.spinics.net/lists/pgsql/msg186919.html
> Row based security and column masking don't work correctly for materialized tables (views)
> ------------------------------------------------------------------------------------------
>
> Key: TEIID-5802
> URL: https://issues.jboss.org/browse/TEIID-5802
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 12.0
> Environment: teiid-12.0.0 on WildFly Full 14.0.1.Final (WildFly Core 6.0.2.Final)
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Blocker
>
> Row based security doesn't work for materialized tables (views). For example, having the following permissions for materialized views.test_view1 and views.test_view2 views according to query plan Teiid applies ApplySecurity rule only for views.test_view2 view but should also apply for views.test_view1 view:
> {code:xml}
> <data-role name="role1" any-authenticated="true" allow-create-temporary-tables="true">
> <description>Allow read only</description>
> <permission>
> <resource-name>dsp</resource-name>
> <allow-read>true</allow-read>
> </permission>
> <permission>
> <resource-name>views.test_view1</resource-name>
> <allow-read>true</allow-read>
> <condition constraint="false">col2 > 1</condition>
> </permission>
> <permission>
> <resource-name>views.test_view2</resource-name>
> <allow-read>true</allow-read>
> <condition constraint="false">col0 = 'sa'</condition>
> </permission>
> </data-role>
> {code}
> The same situation is for column masking feature, see below.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 4 months
[JBoss JIRA] (TEIID-5802) Row based security and column masking don't work correctly for materialized tables (views)
by Dmitrii Pogorelov (Jira)
[ https://issues.jboss.org/browse/TEIID-5802?page=com.atlassian.jira.plugin... ]
Dmitrii Pogorelov commented on TEIID-5802:
------------------------------------------
[~shawkins] are you planning to use the legacy feature of scoping materialized views to a user or not to change anything at all? Any estimations?
> Row based security and column masking don't work correctly for materialized tables (views)
> ------------------------------------------------------------------------------------------
>
> Key: TEIID-5802
> URL: https://issues.jboss.org/browse/TEIID-5802
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 12.0
> Environment: teiid-12.0.0 on WildFly Full 14.0.1.Final (WildFly Core 6.0.2.Final)
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Blocker
>
> Row based security doesn't work for materialized tables (views). For example, having the following permissions for materialized views.test_view1 and views.test_view2 views according to query plan Teiid applies ApplySecurity rule only for views.test_view2 view but should also apply for views.test_view1 view:
> {code:xml}
> <data-role name="role1" any-authenticated="true" allow-create-temporary-tables="true">
> <description>Allow read only</description>
> <permission>
> <resource-name>dsp</resource-name>
> <allow-read>true</allow-read>
> </permission>
> <permission>
> <resource-name>views.test_view1</resource-name>
> <allow-read>true</allow-read>
> <condition constraint="false">col2 > 1</condition>
> </permission>
> <permission>
> <resource-name>views.test_view2</resource-name>
> <allow-read>true</allow-read>
> <condition constraint="false">col0 = 'sa'</condition>
> </permission>
> </data-role>
> {code}
> The same situation is for column masking feature, see below.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 4 months
[JBoss JIRA] (TEIID-5801) Communications link failure during commit() error message when copying a MySQL table to a DB
by Dmitrii Pogorelov (Jira)
[ https://issues.jboss.org/browse/TEIID-5801?page=com.atlassian.jira.plugin... ]
Dmitrii Pogorelov commented on TEIID-5801:
------------------------------------------
[~shawkins] thx a lot! I've just checked your solution and it worked for my case.
> If that is sufficient for your case, I'd leave the remaining case
ok, at the moment the solution is enough for me.
Thx again!
> Communications link failure during commit() error message when copying a MySQL table to a DB
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-5801
> URL: https://issues.jboss.org/browse/TEIID-5801
> Project: Teiid
> Issue Type: Quality Risk
> Components: Query Engine
> Affects Versions: 12.0
> Environment: teiid-12.0.0 on WildFly Full 14.0.1.Final (WildFly Core 6.0.2.Final)
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Attachments: server_1_fail.log, server_2_works.log, server_teiid.log
>
>
> When copying a MySQL table, for example, to PostgreSQL:
> {code:sql}
> insert into dwh_pg.test_target SELECT * FROM my.test_source ;;
> {code}
> in the end of the process Teiid throws out the following stacktrace (though rows are inserted in PostgreSQL successfully, seems that Teiid can't close read transaction for MySQL):
> {code}
> 2019-08-01 16:48:23,119 WARN [org.jboss.jca.core.connectionmanager.listener.TxConnectionListener] (Worker3_QueryProcessorQueue34) TidBkmeGWJN8 IJ000305: Connection error occured: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@75284e6d[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@5b8d92c7 connection handles=0 lastReturned=1564670796599 lastValidated=1564670796598 lastCheckedOut=1564670796678 trackByTx=true pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@ae21718 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@338041b4[pool=lingoda_read_replica] xaResource=LocalXAResourceImpl@1fcd6b81[connectionListener=75284e6d connectionManager=20f22ec1 warned=false currentXid=null productName=MySQL productVersion=5.6.34-log jndiName=java:/lingoda_read_replica] txSync=TransactionSynchronization@1367866468{tx=Local transaction (delegate=TransactionImple < ac, BasicAction: 0:ffffc0a8008c:33252ff9:5d42fad3:11 status: ActionStatus.PREPARING >, owner=Local transaction context for provider JBoss JTA transaction provider) wasTrackByTx=true enlisted=true cancel=false}]: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1014) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:988) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:974) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919) at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1700) at org.jboss.jca.adapters.jdbc.local.LocalManagedConnection.commit(LocalManagedConnection.java:96) at org.jboss.jca.core.tx.jbossts.LocalXAResourceImpl.commit(LocalXAResourceImpl.java:172) at com.arjuna.ats.internal.jta.resources.arjunacore.XAOnePhaseResource.commit(XAOnePhaseResource.java:120) at com.arjuna.ats.internal.arjuna.abstractrecords.LastResourceRecord.topLevelPrepare(LastResourceRecord.java:152) at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2664) at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2614) at com.arjuna.ats.arjuna.coordinator.BasicAction.prepare(BasicAction.java:2157) at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1503) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1288) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:77) at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71) at org.teiid.dqp.internal.process.TransactionServerImpl.commitDirect(TransactionServerImpl.java:384) at org.teiid.dqp.internal.process.TransactionServerImpl.commit(TransactionServerImpl.java:515) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.teiid.logging.LogManager$LoggingProxy.invoke(LogManager.java:117) at com.sun.proxy.$Proxy25.commit(Unknown Source) at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:514) at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362) at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43) at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285) at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281) at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:113)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:199) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
> {code}
> I tried to reproduce the problem with local MySQL and PostgreSQL but couldn't. The problem can be reproduced only when using remote MySQL and PostgreSQL. On my local machine the error appears with limit more than 200000 rows, on another machines the exception appeared when setting limit 400000 and more. It seems it's related maybe somehow with MySQL timeouts or network delays. If I copy the table from remote MySQL to local PostgeSQL the error doesn't appear, and vice versa, if I copy the table from local MySQL to remote PostgreSQL the error doesn't appear again. I don't have an access to the remote MySQL to have a look at its internal options. I also tried to set net_write_timeout=1800 jdbc property for data source of the remote MySQL, tcpKeepAlive=true, tried to set ThreadBound MySQL translator property to true value - it didn't help at all. What do you think, is it possible to avoid the error on Teiid level?
> I also attached a server log with org.teiid.CONNECTOR and org.teiid.PROCESSOR log outputs.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 4 months