[JBoss JIRA] (TEIID-4745) Avoid loading the same file over again when using 'org.jboss.teiid.resource-adapter.file'
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4745?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4745.
-----------------------------------
Resolution: Done
Added a MATVIEW_POLLING_QUERY property and exposed the MATVIEW_POLLING_INTERVAL from TEIID-5315. These are general enough to be applicable to file processing and other scenarios.
> Avoid loading the same file over again when using 'org.jboss.teiid.resource-adapter.file'
> -----------------------------------------------------------------------------------------
>
> Key: TEIID-4745
> URL: https://issues.jboss.org/browse/TEIID-4745
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Environment: * WildFly 10
> * Teiid Server 9.1.2
> * CentOs 7
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> When using the file resource adapter _org.jboss.teiid.resource-adapter.file_, it should be possible that the system per se, by the means of a new property in the resource-adapter, or other option in the view definition, or other configuration elsewhere, to not to unnecessary load a file that is already loaded. Theoretically it should be possible, for example, to check the file date or the file checksum, to see if the file is already loaded by _Teiid_. If it is, then there is no need to parse the file again.
> This is important due to, specially, performance improvements. If there are a lot of files to be processed, and these files are big (200 MB or more), the system will be processing over and over again the same files if they are not updated for quite some time.
> If, for example, the user uses a Materialized View, then the view will be constantly being updated with the same data, which brings another performance penalty.
> *Note*: This could be extended to other resource-adapters with some sort of last data date for example, if the user whant to use this sort of mechanism. For example, if using a database like MySQL as source, then tthe user could define a last update table, and the system could do a check first against this table before tryng to load the data.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 8 months
[JBoss JIRA] (TEIID-5318) MaterializtionManager probe can be too rapid.
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-5318:
-------------------------------------
Summary: MaterializtionManager probe can be too rapid.
Key: TEIID-5318
URL: https://issues.jboss.org/browse/TEIID-5318
Project: Teiid
Issue Type: Quality Risk
Components: Server
Reporter: Steven Hawkins
Fix For: 10.x
If a ttl load is in progress the probe is repeated every minute - regardless of how much time is remaining. This can happen due to manual loads or in clusters where members start the materialization jobs with some skew. While this is not harmful, it does lead to a lot of logging and queries against the status table which could be avoided.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 8 months
[JBoss JIRA] (TEIID-5270) Materialization translator infinispan-hotrod inconsistent state
by Johnathon Lee (JIRA)
[ https://issues.jboss.org/browse/TEIID-5270?page=com.atlassian.jira.plugin... ]
Johnathon Lee updated TEIID-5270:
---------------------------------
Fix Version/s: (was: 8.12.13.6_4)
> Materialization translator infinispan-hotrod inconsistent state
> ---------------------------------------------------------------
>
> Key: TEIID-5270
> URL: https://issues.jboss.org/browse/TEIID-5270
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.12.6_4
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Critical
>
> There is an inconsistency between contents of status table and jdg cache when using infinispan-hotrod translator for materialization.
> For a vdb:
> {code:xml}
> <vdb name="teiid5259" version="1">
> <model name="source">
> <source connection-jndi-name="java:/jdg71HotrodDS" name="source"
> translator-name="tr-override" />
> <metadata type="DDL">
> <![CDATA[
> CREATE FOREIGN TABLE a(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${primary.cache.name}');
> CREATE FOREIGN TABLE a_stage(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${staging.cache.name}');
> ]]>
> </metadata>
> <metadata type="NATIVE" />
> </model>
> <model name="h2Source">
> <source name="h2Source" translator-name="h2" connection-jndi-name="java:/h2-localhost-9292-ds" />
> </model>
> <model name="View" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW mat_a(
> id integer NOT NULL
> ) OPTIONS (MATERIALIZED 'TRUE',
> MATERIALIZED_TABLE 'source.a',
> "teiid_rel:MATERIALIZED_STAGE_TABLE" 'source.a_stage',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'execute source.native(''rename source.a_stage source.a'');',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute source.native(''truncate source.a_stage'');',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'THROW_EXCEPTION',
> "teiid_rel:MATVIEW_TTL" 10000
> )
> AS SELECT 1 UNION ALL SELECT 2;
> ]]>
> </metadata>
> </model>
> <translator name="tr-override" type="infinispan-hotrod">
> <property name="SupportsDirectQueryProcedure" value="true"/>
> </translator>
> </vdb>
> {code}
> I can reach a state when:
> # The status table states LOADED for the view
> # The DATA SRC commands inserting into JDG ended (according to teiid-command log)
> # But consecutive query against the view returns lower count (even though having THROW_EXCEPTION in ONERROR_ACTION property)
> See the attached command log:
> {code}
> 13:59:31,679 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) L8MeUp92Tess START USER COMMAND: startTime=2018-03-01 13:59:31.679 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess applicationName=internal principal=embedded-async vdbName=teiid5259 vdbVersion=1 sql=execute SYSADMIN.loadMatView('View','mat_a')
> 13:59:31,680 INFO [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue14) SBqGYWW07OB5 END USER COMMAND: endTime=2018-03-01 13:59:31.68 requestID=SBqGYWW07OB5.0 txID=null sessionID=SBqGYWW07OB5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=SELECT g_0.NAME, g_0.TARGETSCHEMANAME, g_0.TARGETNAME, g_0.VALID, g_0.LOADSTATE, g_0.UPDATED, g_0.CARDINALITY, g_0.LOADNUMBER FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[SELECT g_0."NAME", g_0."TARGETSCHEMANAME", g_0."TARGETNAME", g_0."VALID", g_0."LOADSTATE", g_0."UPDATED", g_0."CARDINALITY", g_0."LOADNUMBER" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:31,716 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue17) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.716 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1071137
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADING', valid = FALSE, Updated = {ts'2018-03-01 13:59:31.719'}, Cardinality = -1 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADING', "VALID" = 0, "UPDATED" = TIMESTAMP '2018-03-01 13:59:31.719', "CARDINALITY" = -1 WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:31,728 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.728 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1169356
> 13:59:31,731 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.731 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('truncate source.a_stage')
> 13:59:31,845 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:31.845 requestID=uaySHcXyc0o5.0 sourceCommandID=74 executionID=6 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=3558736
> 13:59:31,847 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.847 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:32,040 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.04 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=1493046
> 13:59:32,047 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.047 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (1)
> 13:59:32,460 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.46 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4783162
> 13:59:32,462 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.462 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (2)
> 13:59:32,753 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.753 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=0 cpuTime(ns)=5021072
> 13:59:32,800 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.8 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 1, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:32.756'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:32,801 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 SOURCE SRC COMMAND: endTime=2018-03-01 13:59:32.801 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 1, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:32.756', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:32,804 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.804 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=31050942
> 13:59:32,806 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END USER COMMAND: endTime=2018-03-01 13:59:32.806 requestID=uaySHcXyc0o5.0 txID=null sessionID=uaySHcXyc0o5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:33,005 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.005 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4943881
> 13:59:33,014 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.014 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:33,723 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.723 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=4426814
> 13:59:33,731 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.731 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:33.725'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 2)
> 13:59:33,732 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:33.732 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:33.725', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 2]
> 13:59:33,737 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.737 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=3747146
> 13:59:33,740 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END USER COMMAND: endTime=2018-03-01 13:59:33.74 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:35,871 INFO [org.teiid.COMMAND_LOG] (New I/O worker #2) eP8bDNkHEpSG START USER COMMAND: startTime=2018-03-01 13:59:35.871 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG applicationName=JDBC principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 sql=SELECT COUNT(*) FROM mat_a
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.VALID, g_0.LOADSTATE FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG SOURCE SRC COMMAND: endTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sourceCommand=[SELECT g_0."VALID", g_0."LOADSTATE" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:35,878 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue23) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:35.878 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=904441
> 13:59:35,879 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue24) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.879 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.id FROM source.a AS g_0
> 13:59:36,326 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue25) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:36.326 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=32679662
> 13:59:36,384 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue26) eP8bDNkHEpSG END USER COMMAND: endTime=2018-03-01 13:59:36.384 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 8 months
[JBoss JIRA] (TEIID-5270) Materialization translator infinispan-hotrod inconsistent state
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-5270?page=com.atlassian.jira.plugin... ]
Van Halbert updated TEIID-5270:
-------------------------------
Git Pull Request: https://github.com/vhalbert/teiid/pull/1, https://github.com/teiid/jdg/pull/40 (was: https://github.com/vhalbert/teiid/pull/1)
> Materialization translator infinispan-hotrod inconsistent state
> ---------------------------------------------------------------
>
> Key: TEIID-5270
> URL: https://issues.jboss.org/browse/TEIID-5270
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.12.6_4
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Critical
> Fix For: 8.12.13.6_4
>
>
> There is an inconsistency between contents of status table and jdg cache when using infinispan-hotrod translator for materialization.
> For a vdb:
> {code:xml}
> <vdb name="teiid5259" version="1">
> <model name="source">
> <source connection-jndi-name="java:/jdg71HotrodDS" name="source"
> translator-name="tr-override" />
> <metadata type="DDL">
> <![CDATA[
> CREATE FOREIGN TABLE a(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${primary.cache.name}');
> CREATE FOREIGN TABLE a_stage(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${staging.cache.name}');
> ]]>
> </metadata>
> <metadata type="NATIVE" />
> </model>
> <model name="h2Source">
> <source name="h2Source" translator-name="h2" connection-jndi-name="java:/h2-localhost-9292-ds" />
> </model>
> <model name="View" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW mat_a(
> id integer NOT NULL
> ) OPTIONS (MATERIALIZED 'TRUE',
> MATERIALIZED_TABLE 'source.a',
> "teiid_rel:MATERIALIZED_STAGE_TABLE" 'source.a_stage',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'execute source.native(''rename source.a_stage source.a'');',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute source.native(''truncate source.a_stage'');',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'THROW_EXCEPTION',
> "teiid_rel:MATVIEW_TTL" 10000
> )
> AS SELECT 1 UNION ALL SELECT 2;
> ]]>
> </metadata>
> </model>
> <translator name="tr-override" type="infinispan-hotrod">
> <property name="SupportsDirectQueryProcedure" value="true"/>
> </translator>
> </vdb>
> {code}
> I can reach a state when:
> # The status table states LOADED for the view
> # The DATA SRC commands inserting into JDG ended (according to teiid-command log)
> # But consecutive query against the view returns lower count (even though having THROW_EXCEPTION in ONERROR_ACTION property)
> See the attached command log:
> {code}
> 13:59:31,679 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) L8MeUp92Tess START USER COMMAND: startTime=2018-03-01 13:59:31.679 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess applicationName=internal principal=embedded-async vdbName=teiid5259 vdbVersion=1 sql=execute SYSADMIN.loadMatView('View','mat_a')
> 13:59:31,680 INFO [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue14) SBqGYWW07OB5 END USER COMMAND: endTime=2018-03-01 13:59:31.68 requestID=SBqGYWW07OB5.0 txID=null sessionID=SBqGYWW07OB5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=SELECT g_0.NAME, g_0.TARGETSCHEMANAME, g_0.TARGETNAME, g_0.VALID, g_0.LOADSTATE, g_0.UPDATED, g_0.CARDINALITY, g_0.LOADNUMBER FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[SELECT g_0."NAME", g_0."TARGETSCHEMANAME", g_0."TARGETNAME", g_0."VALID", g_0."LOADSTATE", g_0."UPDATED", g_0."CARDINALITY", g_0."LOADNUMBER" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:31,716 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue17) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.716 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1071137
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADING', valid = FALSE, Updated = {ts'2018-03-01 13:59:31.719'}, Cardinality = -1 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADING', "VALID" = 0, "UPDATED" = TIMESTAMP '2018-03-01 13:59:31.719', "CARDINALITY" = -1 WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:31,728 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.728 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1169356
> 13:59:31,731 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.731 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('truncate source.a_stage')
> 13:59:31,845 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:31.845 requestID=uaySHcXyc0o5.0 sourceCommandID=74 executionID=6 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=3558736
> 13:59:31,847 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.847 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:32,040 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.04 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=1493046
> 13:59:32,047 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.047 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (1)
> 13:59:32,460 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.46 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4783162
> 13:59:32,462 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.462 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (2)
> 13:59:32,753 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.753 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=0 cpuTime(ns)=5021072
> 13:59:32,800 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.8 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 1, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:32.756'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:32,801 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 SOURCE SRC COMMAND: endTime=2018-03-01 13:59:32.801 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 1, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:32.756', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:32,804 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.804 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=31050942
> 13:59:32,806 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END USER COMMAND: endTime=2018-03-01 13:59:32.806 requestID=uaySHcXyc0o5.0 txID=null sessionID=uaySHcXyc0o5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:33,005 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.005 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4943881
> 13:59:33,014 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.014 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:33,723 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.723 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=4426814
> 13:59:33,731 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.731 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:33.725'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 2)
> 13:59:33,732 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:33.732 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:33.725', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 2]
> 13:59:33,737 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.737 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=3747146
> 13:59:33,740 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END USER COMMAND: endTime=2018-03-01 13:59:33.74 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:35,871 INFO [org.teiid.COMMAND_LOG] (New I/O worker #2) eP8bDNkHEpSG START USER COMMAND: startTime=2018-03-01 13:59:35.871 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG applicationName=JDBC principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 sql=SELECT COUNT(*) FROM mat_a
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.VALID, g_0.LOADSTATE FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG SOURCE SRC COMMAND: endTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sourceCommand=[SELECT g_0."VALID", g_0."LOADSTATE" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:35,878 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue23) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:35.878 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=904441
> 13:59:35,879 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue24) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.879 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.id FROM source.a AS g_0
> 13:59:36,326 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue25) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:36.326 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=32679662
> 13:59:36,384 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue26) eP8bDNkHEpSG END USER COMMAND: endTime=2018-03-01 13:59:36.384 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 8 months
[JBoss JIRA] (TEIID-5270) Materialization translator infinispan-hotrod inconsistent state
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-5270?page=com.atlassian.jira.plugin... ]
Van Halbert updated TEIID-5270:
-------------------------------
Fix Version/s: 8.12.13.6_4
> Materialization translator infinispan-hotrod inconsistent state
> ---------------------------------------------------------------
>
> Key: TEIID-5270
> URL: https://issues.jboss.org/browse/TEIID-5270
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.12.6_4
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Critical
> Fix For: 8.12.13.6_4
>
>
> There is an inconsistency between contents of status table and jdg cache when using infinispan-hotrod translator for materialization.
> For a vdb:
> {code:xml}
> <vdb name="teiid5259" version="1">
> <model name="source">
> <source connection-jndi-name="java:/jdg71HotrodDS" name="source"
> translator-name="tr-override" />
> <metadata type="DDL">
> <![CDATA[
> CREATE FOREIGN TABLE a(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${primary.cache.name}');
> CREATE FOREIGN TABLE a_stage(id integer NOT NULL PRIMARY KEY) OPTIONS (UPDATABLE TRUE, "teiid_ispn:cache" '${staging.cache.name}');
> ]]>
> </metadata>
> <metadata type="NATIVE" />
> </model>
> <model name="h2Source">
> <source name="h2Source" translator-name="h2" connection-jndi-name="java:/h2-localhost-9292-ds" />
> </model>
> <model name="View" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW mat_a(
> id integer NOT NULL
> ) OPTIONS (MATERIALIZED 'TRUE',
> MATERIALIZED_TABLE 'source.a',
> "teiid_rel:MATERIALIZED_STAGE_TABLE" 'source.a_stage',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'execute source.native(''rename source.a_stage source.a'');',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute source.native(''truncate source.a_stage'');',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'THROW_EXCEPTION',
> "teiid_rel:MATVIEW_TTL" 10000
> )
> AS SELECT 1 UNION ALL SELECT 2;
> ]]>
> </metadata>
> </model>
> <translator name="tr-override" type="infinispan-hotrod">
> <property name="SupportsDirectQueryProcedure" value="true"/>
> </translator>
> </vdb>
> {code}
> I can reach a state when:
> # The status table states LOADED for the view
> # The DATA SRC commands inserting into JDG ended (according to teiid-command log)
> # But consecutive query against the view returns lower count (even though having THROW_EXCEPTION in ONERROR_ACTION property)
> See the attached command log:
> {code}
> 13:59:31,679 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) L8MeUp92Tess START USER COMMAND: startTime=2018-03-01 13:59:31.679 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess applicationName=internal principal=embedded-async vdbName=teiid5259 vdbVersion=1 sql=execute SYSADMIN.loadMatView('View','mat_a')
> 13:59:31,680 INFO [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue14) SBqGYWW07OB5 END USER COMMAND: endTime=2018-03-01 13:59:31.68 requestID=SBqGYWW07OB5.0 txID=null sessionID=SBqGYWW07OB5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=SELECT g_0.NAME, g_0.TARGETSCHEMANAME, g_0.TARGETNAME, g_0.VALID, g_0.LOADSTATE, g_0.UPDATED, g_0.CARDINALITY, g_0.LOADNUMBER FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:31,715 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue16) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.715 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[SELECT g_0."NAME", g_0."TARGETSCHEMANAME", g_0."TARGETNAME", g_0."VALID", g_0."LOADSTATE", g_0."UPDATED", g_0."CARDINALITY", g_0."LOADNUMBER" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:31,716 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue17) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.716 requestID=L8MeUp92Tess.0 sourceCommandID=70 executionID=8 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1071137
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADING', valid = FALSE, Updated = {ts'2018-03-01 13:59:31.719'}, Cardinality = -1 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:31,725 DEBUG [org.teiid.COMMAND_LOG] (Worker1_QueryProcessorQueue18) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:31.725 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADING', "VALID" = 0, "UPDATED" = TIMESTAMP '2018-03-01 13:59:31.719', "CARDINALITY" = -1 WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:31,728 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:31.728 requestID=L8MeUp92Tess.0 sourceCommandID=71 executionID=9 txID=null modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=1169356
> 13:59:31,731 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.731 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('truncate source.a_stage')
> 13:59:31,845 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:31.845 requestID=uaySHcXyc0o5.0 sourceCommandID=74 executionID=6 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=3558736
> 13:59:31,847 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:31.847 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:32,040 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.04 requestID=L8MeUp92Tess.0 sourceCommandID=73 executionID=10 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=1493046
> 13:59:32,047 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.047 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (1)
> 13:59:32,460 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:32.46 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=12 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4783162
> 13:59:32,462 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.462 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=INSERT INTO source.a_stage (id) VALUES (2)
> 13:59:32,753 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.753 requestID=uaySHcXyc0o5.0 sourceCommandID=79 executionID=11 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=0 cpuTime(ns)=5021072
> 13:59:32,800 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 START DATA SRC COMMAND: startTime=2018-03-01 13:59:32.8 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 1, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:32.756'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 1)
> 13:59:32,801 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 SOURCE SRC COMMAND: endTime=2018-03-01 13:59:32.801 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 1, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:32.756', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 1]
> 13:59:32,804 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END SRC COMMAND: endTime=2018-03-01 13:59:32.804 requestID=uaySHcXyc0o5.0 sourceCommandID=80 executionID=14 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:e status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=uaySHcXyc0o5 principal=embedded-async finalRowCount=1 cpuTime(ns)=31050942
> 13:59:32,806 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue11) uaySHcXyc0o5 END USER COMMAND: endTime=2018-03-01 13:59:32.806 requestID=uaySHcXyc0o5.0 txID=null sessionID=uaySHcXyc0o5 principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:33,005 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.005 requestID=L8MeUp92Tess.0 sourceCommandID=74 executionID=13 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=4943881
> 13:59:33,014 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.014 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async sql=EXEC source.native('rename source.a_stage source.a')
> 13:59:33,723 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue19) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.723 requestID=L8MeUp92Tess.0 sourceCommandID=79 executionID=15 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=source translatorName=tr-override sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=0 cpuTime(ns)=4426814
> 13:59:33,731 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess START DATA SRC COMMAND: startTime=2018-03-01 13:59:33.731 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sql=UPDATE h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE SET LoadNumber = 2, LoadState = 'LOADED', valid = TRUE, Updated = {ts'2018-03-01 13:59:33.725'}, Cardinality = 2 WHERE (VDBName = 'teiid5259') AND (VDBVersion = 1) AND (schemaName = 'View') AND (Name = 'mat_a') AND (loadNumber = 2)
> 13:59:33,732 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess SOURCE SRC COMMAND: endTime=2018-03-01 13:59:33.732 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async sourceCommand=[UPDATE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" SET "LOADNUMBER" = 2, "LOADSTATE" = 'LOADED', "VALID" = 1, "UPDATED" = TIMESTAMP '2018-03-01 13:59:33.725', "CARDINALITY" = ? WHERE "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBNAME" = 'teiid5259' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."VDBVERSION" = 1 AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."SCHEMANAME" = 'View' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."NAME" = 'mat_a' AND "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE"."LOADNUMBER" = 2]
> 13:59:33,737 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END SRC COMMAND: endTime=2018-03-01 13:59:33.737 requestID=L8MeUp92Tess.0 sourceCommandID=80 executionID=16 txID=TransactionImple < ac, BasicAction: 0:ffff0a2804e2:b3ef2e2:5a97f919:f status: ActionStatus.RUNNING > modelName=h2Source translatorName=h2 sessionID=L8MeUp92Tess principal=embedded-async finalRowCount=1 cpuTime(ns)=3747146
> 13:59:33,740 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue20) L8MeUp92Tess END USER COMMAND: endTime=2018-03-01 13:59:33.74 requestID=L8MeUp92Tess.0 txID=null sessionID=L8MeUp92Tess principal=embedded-async vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> 13:59:35,871 INFO [org.teiid.COMMAND_LOG] (New I/O worker #2) eP8bDNkHEpSG START USER COMMAND: startTime=2018-03-01 13:59:35.871 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG applicationName=JDBC principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 sql=SELECT COUNT(*) FROM mat_a
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.VALID, g_0.LOADSTATE FROM h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE AS g_0 WHERE (g_0.VDBNAME = 'teiid5259') AND (g_0.VDBVERSION = 1) AND (g_0.SCHEMANAME = 'View') AND (g_0.NAME = 'mat_a')
> 13:59:35,877 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue22) eP8bDNkHEpSG SOURCE SRC COMMAND: endTime=2018-03-01 13:59:35.877 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security sourceCommand=[SELECT g_0."VALID", g_0."LOADSTATE" FROM "DB"."PUBLIC"."DV_MATVIEWS_STATUSTABLE" AS g_0 WHERE g_0."VDBNAME" = 'teiid5259' AND g_0."VDBVERSION" = 1 AND g_0."SCHEMANAME" = 'View' AND g_0."NAME" = 'mat_a']
> 13:59:35,878 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue23) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:35.878 requestID=eP8bDNkHEpSG.0 sourceCommandID=6 executionID=17 txID=null modelName=h2Source translatorName=h2 sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=904441
> 13:59:35,879 DEBUG [org.teiid.COMMAND_LOG] (Worker2_QueryProcessorQueue24) eP8bDNkHEpSG START DATA SRC COMMAND: startTime=2018-03-01 13:59:35.879 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security sql=SELECT g_0.id FROM source.a AS g_0
> 13:59:36,326 DEBUG [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue25) eP8bDNkHEpSG END SRC COMMAND: endTime=2018-03-01 13:59:36.326 requestID=eP8bDNkHEpSG.0 sourceCommandID=9 executionID=18 txID=null modelName=source translatorName=tr-override sessionID=eP8bDNkHEpSG principal=user@teiid-security finalRowCount=1 cpuTime(ns)=32679662
> 13:59:36,384 INFO [org.teiid.COMMAND_LOG] (Worker0_QueryProcessorQueue26) eP8bDNkHEpSG END USER COMMAND: endTime=2018-03-01 13:59:36.384 requestID=eP8bDNkHEpSG.0 txID=null sessionID=eP8bDNkHEpSG principal=user@teiid-security vdbName=teiid5259 vdbVersion=1 finalRowCount=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 8 months
[JBoss JIRA] (TEIID-5315) MATVIEW_MAX_STALENESS_PCT reloads only based upon ttl
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5315?page=com.atlassian.jira.plugin... ]
Steven Hawkins reopened TEIID-5315:
-----------------------------------
With the TEIID-4283 fix, there was a shift in the functionality of the loadMatView procedure - it no longer was attempting to do a load conditional on ttl - but to always perform the load. This is the same as the refreshMatView for internal. This means that the more frequent scheduling of a task based upon the staleness is simply reloading every time.
> MATVIEW_MAX_STALENESS_PCT reloads only based upon ttl
> -----------------------------------------------------
>
> Key: TEIID-5315
> URL: https://issues.jboss.org/browse/TEIID-5315
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3, 10.1.3, 10.2.1
>
>
> https://github.com/teiid/teiid/blob/6d378d44fc7cb220ebc3a7068ff5f747e49dc...
> Should schedule a recurring task for reloading based upon the staleness percentage, instead it is only a oneTimeJob. This means that after the first load, it will revert to using the ttl for reloads.
> We should also be explicit in the docs that the staleness is checked every minute.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 8 months