[JBoss JIRA] (TEIID-4277) Infinispan Cache DSL translator: materialization incomplete results during view's loading
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4277?page=com.atlassian.jira.plugin... ]
Van Halbert edited comment on TEIID-4277 at 7/19/16 11:17 AM:
--------------------------------------------------------------
Adding an extension property:
{code}
public static final String OBJECT_URI = "{http://www.teiid.org/translator/object/2016}"; //$NON-NLS-1$
@ExtensionMetadataProperty(applicable=Table.class, datatype=String.class, display="Primary Table", description="Indicates the primary table that this staging table is used for", required=true)
public static final String PRIMARY_TABLE_PROPERTY= OBJECT_URI+PRIMARY_TABLE; //$NON-NLS-1$
{code}
Which will result in the following metadata (example): (note the new namespace and the primary_table property)
{code}
SET NAMESPACE 'http://www.teiid.org/translator/object/2016' AS n0;
CREATE FOREIGN TABLE ST_Trade (
ST_TradeObject object OPTIONS (NAMEINSOURCE 'this', SELECTABLE FALSE, UPDATABLE FALSE, SEARCHABLE 'Unsearchable', NATIVE_TYPE 'org.teiid.translator.object.testdata.trades.Trade'),
tradeId long NOT NULL OPTIONS (NAMEINSOURCE 'tradeId', SEARCHABLE 'Searchable', NATIVE_TYPE 'long'),
name string OPTIONS (NAMEINSOURCE 'name', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.lang.String'),
settled boolean OPTIONS (NAMEINSOURCE 'settled', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'boolean'),
tradeDate date OPTIONS (NAMEINSOURCE 'tradeDate', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.util.Date'),
CONSTRAINT PK_TRADEID PRIMARY KEY(tradeId)
) OPTIONS (UPDATABLE TRUE, "n0:primary_table" 'ObjectSchema.Trade');
{code}
was (Author: van.halbert):
Adding an extension property:
public static final String OBJECT_URI = "{http://www.teiid.org/translator/object/2016}"; //$NON-NLS-1$
@ExtensionMetadataProperty(applicable=Table.class, datatype=String.class, display="Primary Table", description="Indicates the primary table that this staging table is used for", required=true)
public static final String PRIMARY_TABLE_PROPERTY= OBJECT_URI+PRIMARY_TABLE; //$NON-NLS-1$
Which will result in the following metadata (example): (note the new namespace and the primary_table property)
SET NAMESPACE 'http://www.teiid.org/translator/object/2016' AS n0;
CREATE FOREIGN TABLE ST_Trade (
ST_TradeObject object OPTIONS (NAMEINSOURCE 'this', SELECTABLE FALSE, UPDATABLE FALSE, SEARCHABLE 'Unsearchable', NATIVE_TYPE 'org.teiid.translator.object.testdata.trades.Trade'),
tradeId long NOT NULL OPTIONS (NAMEINSOURCE 'tradeId', SEARCHABLE 'Searchable', NATIVE_TYPE 'long'),
name string OPTIONS (NAMEINSOURCE 'name', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.lang.String'),
settled boolean OPTIONS (NAMEINSOURCE 'settled', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'boolean'),
tradeDate date OPTIONS (NAMEINSOURCE 'tradeDate', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.util.Date'),
CONSTRAINT PK_TRADEID PRIMARY KEY(tradeId)
) OPTIONS (UPDATABLE TRUE, "n0:primary_table" 'ObjectSchema.Trade');
> Infinispan Cache DSL translator: materialization incomplete results during view's loading
> -----------------------------------------------------------------------------------------
>
> Key: TEIID-4277
> URL: https://issues.jboss.org/browse/TEIID-4277
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 9.x, 8.12.5
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Blocker
> Fix For: 8.12.5
>
> Attachments: test.log
>
>
> When user tries to query materialized view in a moment when its loading is in progress, incomplete results are returned. The status of the view in corresponding status table is LOADING.
> Vdb used:
> {code:xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <vdb name="jdg-remote" version="1">
> <description>For testing of materialized views</description>
> <property name ="lib" value ="org.jboss.qe.jdg.pojos"></property>
> <model name="jdgSource">
> <source name="jdgSource" translator-name="jdg-override"
> connection-jndi-name="java:/jdg-mat-ds" />
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE CustomerReport(
> customerId integer not null OPTIONS (SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> totalAmount integer,
> CONSTRAINT PK_ID PRIMARY KEY(customerId)
> ) OPTIONS (NAMEINSOURCE '${db.table.prefix}jdg-mat', UPDATABLE TRUE);]]>
> </metadata>
> </model>
> <model name="h2Source">
> <source name="h2Source" translator-name="h2" connection-jndi-name="java:/h2-ds" />
> </model>
> <model name="View" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW external_long_ttl (
> customerId integer NOT NULL,
> totalAmount integer
> ) OPTIONS (MATERIALIZED 'TRUE',
> MATERIALIZED_TABLE 'jdgSource.CustomerReport',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO jdgSource.CustomerReport(customerId,totalAmount) SELECT c.id AS customerId, CONVERT(SUM(o.amount),integer) AS totalAmount FROM DV_MATVIEWS_CUSTOMERS c INNER JOIN DV_MATVIEWS_ORDERS o ON c.id = o.customer_id GROUP BY c.id;',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute jdgSource.native(''truncate cache'');',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'execute jdgSource.native(''swap cache names'');',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
> "teiid_rel:MATVIEW_TTL" 10000
> )
> AS SELECT c.id AS customerId, CONVERT(SUM(o.amount),integer) AS totalAmount FROM DV_MATVIEWS_CUSTOMERS c INNER JOIN DV_MATVIEWS_ORDERS o ON c.id = o.customer_id GROUP BY c.id;
> ]]>
> </metadata>
> </model>
> <translator name="jdg-override" type="infinispan-cache-dsl">
> <property name="SupportsDirectQueryProcedure" value="true" />
> </translator>
> </vdb>
> {code}
> Resource-adapter used:
> {code:xml}
> <resource-adapter id="jdg-mat">
> <module slot="main" id="org.jboss.teiid.resource-adapter.infinispan.dsl"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.infinispan.dsl.InfinispanManagedConnectionFactory" jndi-name="java:/jdg-mat-ds" enabled="true" pool-name="jdg-mat">
> <config-property name="ProtobufDefinitionFile">
> /org/jboss/qe/jdg/remote/protobuf/CustomerReport.proto
> </config-property>
> <config-property name="AliasCacheName">
> ${db.table.prefix}jdg-alias
> </config-property>
> <config-property name="MessageMarshallers">
> org.jboss.qe.jdg.pojo.CustomerReport:org.jboss.qe.jdg.remote.marshaller.CustomerReportMarshaller
> </config-property>
> <config-property name="RemoteServerList">
> [remote-server-address]:11322
> </config-property>
> <config-property name="StagingCacheName">
> ${db.table.prefix}jdg-mat-stage
> </config-property>
> <config-property name="CacheTypeMap">
> ${db.table.prefix}jdg-mat:org.jboss.qe.jdg.pojo.CustomerReport;customerId
> </config-property>
> <config-property name="MessageDescriptor">
> org.jboss.qe.jdg.remote.protobuf.CustomerReport
> </config-property>
> <config-property name="Module">
> org.jboss.qe.jdg.pojos
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> Datasource used (tables that the view queries):
> {code:xml}
> <xa-datasource jndi-name="java:/h2-ds" pool-name="h2-xa-ds" enabled="true">
> <xa-datasource-property name="URL">
> jdbc:h2:tcp://localhost:9292/db;TRACE_LEVEL_FILE=4
> </xa-datasource-property>
> <driver>h2</driver>
> <security>
> <user-name>sa</user-name>
> </security>
> </xa-datasource>
> {code}
> Query invoked to check current contents of materialized view:
> {code:sql}
> SELECT COUNT(*) FROM external_long_ttl;
> {code}
> And expected result is the same as count of DV_MATVIEWS_CUSTOMERS table from source h2Source. In my case 10.
> But when I print out the result I get all numbers from 0 to 10 . See attached log output (search for 'check-count:')
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4283) External Materialization interleaving loads SYSADMIN.loadMatView
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4283?page=com.atlassian.jira.plugin... ]
Steven Hawkins closed TEIID-4283.
---------------------------------
> External Materialization interleaving loads SYSADMIN.loadMatView
> ----------------------------------------------------------------
>
> Key: TEIID-4283
> URL: https://issues.jboss.org/browse/TEIID-4283
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.5
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Priority: Blocker
> Fix For: 9.1, 8.12.5, 9.0.1
>
> Attachments: jdg-log.txt
>
>
> There is a possibility of two concurrent loads of single materialized view when using SYSADMIN.loadMatView function. I observed the issue while:
> # Waiting for ttl-driven reload and in right time I triggered another load explicitly by invoking the loadMatView function.
> # Invoking SYSADMIN.loadMatView function multiple times in quick succession.
> The view definition:
> {code:sql}
> CREATE VIEW external_long_ttl (
> customer_id integer NOT NULL,
> total_amount integer
> ) OPTIONS (MATERIALIZED 'TRUE', UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'Source.JSTASTNY.dv_matviews_mat_view',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'Source.JSTASTNY.dv_matviews_statustable',
> "teiid_rel:ON_VDB_START_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_create) SELECT id, vdb_create+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:ON_VDB_DROP_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_drop) SELECT id, vdb_drop+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO dv_matviews_mat_view_stage(customer_id,total_amount) SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'exec Source.native(''truncate table ${db.table.prefix}dv_matviews_mat_view_stage'');MERGE INTO dv_matviews_check_table(id,before_load) SELECT id, before_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'exec Source.native(''RENAME TABLE ${db.table.prefix}dv_matviews_mat_view_stage TO ${db.table.prefix}dv_matviews_mat_view_temp'');exec Source.native(''RENAME TABLE ${db.table.prefix}dv_matviews_mat_view TO ${db.table.prefix}dv_matviews_mat_view_stage'');exec Source.native(''RENAME TABLE ${db.table.prefix}dv_matviews_mat_view_temp TO ${db.table.prefix}dv_matviews_mat_view'');MERGE INTO dv_matviews_check_table(id,after_load) SELECT id, after_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
> "teiid_rel:MATVIEW_TTL" 20000)
> AS SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;
> {code}
> Query to explicitly load the view:
> {code:sql}
> exec SYSADMIN.loadMatView(schemaName=>'View',viewname=>'external_long_ttl', invalidate=>'true')
> {code}
> I attached a log which starts with ttl-driven load, then the explicit load is performed. The issue noticed is in teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT , where primary and staging tables are being swapped. The second load (explicit) can't find the original table, as the ttl-driven load had already renamed it.
> But this situation of two concurrent loads should never occur.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4343) Impala Translator - With Clause Support
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4343?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4343:
----------------------------------
Issue Type: Bug (was: Feature Request)
> Impala Translator - With Clause Support
> ---------------------------------------
>
> Key: TEIID-4343
> URL: https://issues.jboss.org/browse/TEIID-4343
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.13.3
> Reporter: Scott Wallace
> Assignee: Steven Hawkins
> Fix For: 9.1
>
>
> I'm getting an error using the With clause in Teiid connecting to Impala
> {code}
> with t as (select * from schema_name.table_name)
> select * from t;
> {code}
> Error:
> {code}
> org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidProcessingException: TEIID30504 warehouse: 11300 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: WITH t (col1, col2, col3) AS (SELECT g_0.`col1`, g_0.`col2`, g_0.`col3` FROM schema_name.table_name g_0) SELECT g_0.col1, g_0.col2, g_0.col3 FROM t g_0]
> Elapsed Time: 0 hr, 0 min, 0 sec, 841 ms.
> {code}
> Full error stack:
> {code}
> 14:37:35,491 INFO [org.teiid.CONNECTOR] (Worker1_QueryProcessorQueue1) ImpalaExecutionFactory Commit=true;DatabaseProductName=Impala;DatabaseProductVersion=2.5.0-cdh5.7.0;DriverMajorVersion=2;DriverMajorVersion=5;DriverName=ImpalaJDBC;DriverVersion=02.05.31.1051;IsolationLevel=1
> 14:37:35,516 WARN [org.teiid.CONNECTOR] (Worker1_QueryProcessorQueue1) Connector worker process failed for atomic-request=u1RZzQGciJra.0.1.0: org.teiid.translator.jdbc.JDBCExecutionException: 11300 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: WITH t (col1, col2, col3) AS (SELECT g_0.`col1`, g_0.`col2`, g_0.`col3` FROM schema_name.table_name g_0) SELECT g_0.col1, g_0.col2, g_0.col3 FROM t g_0]
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:131) [translator-jdbc-8.13.3.jar:8.13.3]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:359)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_91]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_91]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_91]
> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_91]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy29.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_91]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_91]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_91]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_91]
> Caused by: java.sql.SQLDataException: [Simba][JDBC](11300) A ResultSet was expected but not generated from query "WITH t (col1, col2, col3) AS (SELECT g_0.`col1`, g_0.`col2`, g_0.`col3` FROM schema_name.table_name g_0) SELECT g_0.col1, g_0.col2, g_0.col3 FROM t g_0". Query not executed.
> at com.cloudera.exceptions.ExceptionConverter.toSQLException(Unknown Source)
> at com.cloudera.jdbc.common.SStatement.checkCondition(Unknown Source)
> at com.cloudera.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
> at com.cloudera.jdbc.common.SPreparedStatement.executeQuery(Unknown Source)
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:504)
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:123) [translator-jdbc-8.13.3.jar:8.13.3]
> ... 18 more
> {code}
> Taking the prepared statement and executing against Impala directly, succeeds without any issues.
> Is the With clause supported in the Impala translator?
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4296) File translator cannot find a file without an extensioin
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4296?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-4296:
-------------------------------------
Assignee: Steven Hawkins
> File translator cannot find a file without an extensioin
> --------------------------------------------------------
>
> Key: TEIID-4296
> URL: https://issues.jboss.org/browse/TEIID-4296
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.12.5
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 9.1
>
>
> Spark creates files with no extension, so using the following:
> EXEC Spark.getTextFiles('part-00001') will generate an error when trying any of the following ways: part-*.* or part*.* or part-*
> org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidProcessingException: TEIID30504 Spark: File not found part*.*
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4296) File translator cannot find a file without an extensioin
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4296?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4296.
-----------------------------------
Resolution: Done
Changed any \* to be treated like a glob wildcard.
> File translator cannot find a file without an extensioin
> --------------------------------------------------------
>
> Key: TEIID-4296
> URL: https://issues.jboss.org/browse/TEIID-4296
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.12.5
> Reporter: Van Halbert
> Fix For: 9.1
>
>
> Spark creates files with no extension, so using the following:
> EXEC Spark.getTextFiles('part-00001') will generate an error when trying any of the following ways: part-*.* or part*.* or part-*
> org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidProcessingException: TEIID30504 Spark: File not found part*.*
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4283) External Materialization interleaving loads SYSADMIN.loadMatView
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-4283?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-4283:
------------------------------------------------
Jan Stastny <jstastny(a)redhat.com> changed the Status of [bug 1348466|https://bugzilla.redhat.com/show_bug.cgi?id=1348466] from ON_QA to VERIFIED
> External Materialization interleaving loads SYSADMIN.loadMatView
> ----------------------------------------------------------------
>
> Key: TEIID-4283
> URL: https://issues.jboss.org/browse/TEIID-4283
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.5
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Priority: Blocker
> Fix For: 9.1, 8.12.5, 9.0.1
>
> Attachments: jdg-log.txt
>
>
> There is a possibility of two concurrent loads of single materialized view when using SYSADMIN.loadMatView function. I observed the issue while:
> # Waiting for ttl-driven reload and in right time I triggered another load explicitly by invoking the loadMatView function.
> # Invoking SYSADMIN.loadMatView function multiple times in quick succession.
> The view definition:
> {code:sql}
> CREATE VIEW external_long_ttl (
> customer_id integer NOT NULL,
> total_amount integer
> ) OPTIONS (MATERIALIZED 'TRUE', UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'Source.JSTASTNY.dv_matviews_mat_view',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'Source.JSTASTNY.dv_matviews_statustable',
> "teiid_rel:ON_VDB_START_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_create) SELECT id, vdb_create+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:ON_VDB_DROP_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_drop) SELECT id, vdb_drop+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO dv_matviews_mat_view_stage(customer_id,total_amount) SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'exec Source.native(''truncate table ${db.table.prefix}dv_matviews_mat_view_stage'');MERGE INTO dv_matviews_check_table(id,before_load) SELECT id, before_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'exec Source.native(''RENAME TABLE ${db.table.prefix}dv_matviews_mat_view_stage TO ${db.table.prefix}dv_matviews_mat_view_temp'');exec Source.native(''RENAME TABLE ${db.table.prefix}dv_matviews_mat_view TO ${db.table.prefix}dv_matviews_mat_view_stage'');exec Source.native(''RENAME TABLE ${db.table.prefix}dv_matviews_mat_view_temp TO ${db.table.prefix}dv_matviews_mat_view'');MERGE INTO dv_matviews_check_table(id,after_load) SELECT id, after_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
> "teiid_rel:MATVIEW_TTL" 20000)
> AS SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;
> {code}
> Query to explicitly load the view:
> {code:sql}
> exec SYSADMIN.loadMatView(schemaName=>'View',viewname=>'external_long_ttl', invalidate=>'true')
> {code}
> I attached a log which starts with ttl-driven load, then the explicit load is performed. The issue noticed is in teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT , where primary and staging tables are being swapped. The second load (explicit) can't find the original table, as the ttl-driven load had already renamed it.
> But this situation of two concurrent loads should never occur.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4213) Cassandra translator: enable pushdown of aggregate functions
by JieRen JieRen (JIRA)
[ https://issues.jboss.org/browse/TEIID-4213?page=com.atlassian.jira.plugin... ]
JieRen JieRen reassigned TEIID-4213:
------------------------------------
Assignee: JieRen JieRen
> Cassandra translator: enable pushdown of aggregate functions
> ------------------------------------------------------------
>
> Key: TEIID-4213
> URL: https://issues.jboss.org/browse/TEIID-4213
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.12.x
> Reporter: Jan Stastny
> Assignee: JieRen JieRen
> Priority: Minor
> Fix For: Open To Community, 9.1
>
>
> Cassandra supports basic aggregate functions:
> MIN(),MAX(),SUM() and AVG()
> Currently Teiid doesn't push such queries.
> NOTE:In case of AVG() there is difference between the values returned from Teiid vs. Cassandra for an integer column. Cassandra returns only whole part of the number as integer, Teiid returns bigdecimal.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (TEIID-4297) Add UDF functions based on OSDQ for POC
by JieRen JieRen (JIRA)
[ https://issues.jboss.org/browse/TEIID-4297?page=com.atlassian.jira.plugin... ]
JieRen JieRen resolved TEIID-4297.
----------------------------------
Resolution: Done
> Add UDF functions based on OSDQ for POC
> ---------------------------------------
>
> Key: TEIID-4297
> URL: https://issues.jboss.org/browse/TEIID-4297
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Ramesh Reddy
> Assignee: JieRen JieRen
> Fix For: 9.1
>
>
> Please add following UDF functions to Teiid library based on maven module
> http://search.maven.org/#search%7Cga%7C1%7Carrahtec
> The UDF functions needs to from this class below
> {code}
> public class Maskutil {
> /**
> * @param a
> * The string that need to randomize
> * vivek singh' will become 'ihg vkeivh'
> */
> public static String toRandomValue(String a) {
> return ShuffleRTM.shuffleString(a);
> }
> /**
> * @param a
> * This function will retrun MD5 hashcode of the string
> * @return String
> */
> public static String toHashValue(String a) {
> if (a == null)
> return "d41d8cd98f00b204e9800998ecf8427e"; // null MD5 value
> return ResultsetToRTM.getMD5(a).toString();
> }
> /**
> * @param a
> * This function will return digit characters of the string
> * @return
> *
> */
> public static String toDigitValue(String a) {
> return StringCaseFormatUtil.digitString(a);
> }
> }
> {code}
> for example Teiid should have UDF functions like
> {code}
> string random(string)
> string hash(string)
> string digit(string)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months