[JBoss JIRA] (TEIID-5090) Issues with matview scripts
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5090?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-5090:
-------------------------------------
Assignee: Steven Hawkins (was: Van Halbert)
> Issues with matview scripts
> ---------------------------
>
> Key: TEIID-5090
> URL: https://issues.jboss.org/browse/TEIID-5090
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.x-6.4
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.0, 9.3.4
>
>
> There are a couple of issues with the script logic. The first is the use of the replace function in the update criteria for updateMatView - this is not generally safe. There are also some checks for internal materialization that look for a null table in the system table, but it's actually the schema that will be null.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-5027) Translator infinispan-hotrod incorrect data type import
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-5027?page=com.atlassian.jira.plugin... ]
Van Halbert updated TEIID-5027:
-------------------------------
Fix Version/s: 8.12.x-6.4
> Translator infinispan-hotrod incorrect data type import
> -------------------------------------------------------
>
> Key: TEIID-5027
> URL: https://issues.jboss.org/browse/TEIID-5027
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
> Priority: Blocker
> Fix For: 10.0, 8.12.x-6.4
>
>
> I have a vdb:
> {code:xml|title=jdg71_crud-vdb.xml}
> <vdb name="jdg71_crud" version="1">
> <model name="Source" type="PHYSICAL" visible="true">
> <source name="jdg7-source" translator-name="infinispan-hotrod" connection-jndi-name="java:/jdg71HotrodDS" />
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE SmallA (
> IntKey integer,
> IntNum integer,
> DoubleNum double,
> ObjectValue object,
> BigDecimalValue bigdecimal,
> BigIntegerValue biginteger,
> CharValue char,
> StringNum string,
> StringKey string PRIMARY KEY,
> FloatNum float,
> LongNum long,
> TimeValue time,
> ShortValue short,
> ByteNum byte,
> TimeStampValue timestamp,
> BooleanValue boolean,
> DateValue date) OPTIONS(UPDATABLE true, "teiid_ispn:cache" '${jdg.cache.name}');
> ]]>
> </metadata>
> <metadata type = "NATIVE"/>
> </model>
> </vdb>
> {code}
> But the resulting metadata are:
> || column_name | column_type ||
> || IntKey | integer ||
> || IntNum | integer ||
> || DoubleNum | double ||
> || ObjectValue | string ||
> || BigDecimalValue | varbinary ||
> || BigIntegerValue | varbinary ||
> || CharValue | string ||
> || StringNum | string ||
> || StringKey | string ||
> || FloatNum | float ||
> || LongNum | long ||
> || TimeValue | varbinary ||
> || ShortValue | integer ||
> || ByteNum | integer ||
> || TimeStampValue | varbinary ||
> || BooleanValue | boolean ||
> || DateValue | varbinary ||
> All the columns should be replaced by appropriate data type as defined in the vdb.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-5090) Issues with matview scripts
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-5090?page=com.atlassian.jira.plugin... ]
Van Halbert reassigned TEIID-5090:
----------------------------------
Assignee: Van Halbert (was: Steven Hawkins)
> Issues with matview scripts
> ---------------------------
>
> Key: TEIID-5090
> URL: https://issues.jboss.org/browse/TEIID-5090
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.x-6.4
> Reporter: Steven Hawkins
> Assignee: Van Halbert
> Fix For: 10.0, 9.3.4
>
>
> There are a couple of issues with the script logic. The first is the use of the replace function in the update criteria for updateMatView - this is not generally safe. There are also some checks for internal materialization that look for a null table in the system table, but it's actually the schema that will be null.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-5077) Couchbase strange behaviour for long numbers
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5077?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5077:
---------------------------------------
> Steven Hawkins What are the expectations for predefined DDL metadata by user? Should the property override the behaviour also in that case?
No, it would not fix this scenario. Long is effectively a non-supported numeric source type, especially if you indicate useDouble. So it is not expected that you would do this. You can either accept precision limitation or use the string source type and a convert at the Teiid layer. The issue with the latter approach is of course performance if you need to do any searching over the field as it will have to be done at the Teiid level.
> Couchbase strange behaviour for long numbers
> --------------------------------------------
>
> Key: TEIID-5077
> URL: https://issues.jboss.org/browse/TEIID-5077
> Project: Teiid
> Issue Type: Bug
> Components: Documentation, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 10.0, 8.12.x-6.4, 9.3.4
>
>
> There is a strange limitation for storing large numbers (e.g. modelled as long) in Couchbase.
> https://forums.couchbase.com/t/strange-behavior-when-saving-big-long-numb...
> The same behaviour with value: 28341791043587382
> which is being stored and retreived as: 28341791043587384
> I feel we need to document this behaviour and provide hints how to overcome it. (Modelling of the column as string on Couchbase's side and defining a view).
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-5077) Couchbase strange behaviour for long numbers
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-5077?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-5077:
------------------------------------
[~shawkins] What are the expectations for predefined DDL metadata by user? Should the property override the behaviour also in that case?
When I use DDL metadata:
{code:sql|title=Source}
CREATE FOREIGN TABLE SmallA (
documentID string PRIMARY KEY,
type string OPTIONS (NAMEINSOURCE '`type`'),
IntKey integer OPTIONS (NAMEINSOURCE '`IntKey`'),
FloatNum double OPTIONS (NAMEINSOURCE '`FloatNum`'),
BigIntegerValue biginteger OPTIONS (NAMEINSOURCE '`BigIntegerValue`'),
StringKey string OPTIONS (NAMEINSOURCE '`StringKey`'),
CharValue string OPTIONS (NAMEINSOURCE '`CharValue`'),
LongNum long OPTIONS (NAMEINSOURCE '`LongNum`'),
DoubleNum double OPTIONS (NAMEINSOURCE '`DoubleNum`'),
ObjectValue object OPTIONS (NAMEINSOURCE '`ObjectValue`'),
ShortValue integer OPTIONS (NAMEINSOURCE '`ShortValue`'),
BigDecimalValue bigdecimal OPTIONS (NAMEINSOURCE '`BigDecimalValue`'),
DateValue string OPTIONS (NAMEINSOURCE '`DateValue`'),
BooleanValue boolean OPTIONS (NAMEINSOURCE '`BooleanValue`'),
TimestampValue string OPTIONS (NAMEINSOURCE '`TimestampValue`'),
ByteNum integer OPTIONS (NAMEINSOURCE '`ByteNum`'),
StringNum string OPTIONS (NAMEINSOURCE '`StringNum`'),
TimeValue string OPTIONS (NAMEINSOURCE '`TimeValue`'),
IntNum integer OPTIONS (NAMEINSOURCE '`IntNum`')
) OPTIONS (NAMEINSOURCE '`dvqe_crud`', UPDATABLE TRUE, "teiid_couchbase:ISARRAYTABLE" 'false', "teiid_couchbase:NAMEDTYPEPAIR" '`type`:''nullSmallA''');
{code}
translator override:
{code:xml}
<translator name="couchbase-override" type="couchbase">
<property name="UseDouble" value="true"/>
</translator>
{code}
and perform an INSERT:
{code:sql}
INSERT INTO CRUD.SmallA
(IntKey, StringKey, IntNum, StringNum, FloatNum, LongNum, DoubleNum, ByteNum, DateValue, TimeValue, TimestampValue, BooleanValue, CharValue, ShortValue, BigIntegerValue, BigDecimalValue, ObjectValue)
VALUES
(10, '10', -2185591, null, convert(10816.7, float), 4964473766577711554, convert(1509403.57, double), null, {d '2008-10-14'}, {t '06:40:48'}, null, false, convert('2', char), convert(9885, short), 18812, null, '-162')
{code}
this is what I can see in couchbase by using cbq cli tool:
{code:json}
{
"dvqe_crud": {
"BigDecimalValue": null,
"BigIntegerValue": 18812,
"BooleanValue": false,
"ByteNum": null,
"CharValue": "2",
"DateValue": "2008-10-14",
"DoubleNum": 1509403.57,
"FloatNum": 10816.7001953125,
"IntKey": 10,
"IntNum": -2185591,
"LongNum": 4964473766577711000,
"ObjectValue": "-162",
"ShortValue": 9885,
"StringKey": "10",
"StringNum": null,
"TimeValue": "06:40:48",
"TimestampValue": null,
"type": "nullSmallA"
}
}
{code}
Shouldn't have the property fixed this?
Moreover, when I use WHERE clause on LongNum, then all the 4964473766577711XXX values match each other.
> Couchbase strange behaviour for long numbers
> --------------------------------------------
>
> Key: TEIID-5077
> URL: https://issues.jboss.org/browse/TEIID-5077
> Project: Teiid
> Issue Type: Bug
> Components: Documentation, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 10.0, 8.12.x-6.4, 9.3.4
>
>
> There is a strange limitation for storing large numbers (e.g. modelled as long) in Couchbase.
> https://forums.couchbase.com/t/strange-behavior-when-saving-big-long-numb...
> The same behaviour with value: 28341791043587382
> which is being stored and retreived as: 28341791043587384
> I feel we need to document this behaviour and provide hints how to overcome it. (Modelling of the column as string on Couchbase's side and defining a view).
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-4208) Increase plan parallelism
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4208?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4208.
-----------------------------------
Resolution: Partially Completed
Since there is greater risk to adding parallelism directly to branching points that will not be included with this issue.
> Increase plan parallelism
> -------------------------
>
> Key: TEIID-4208
> URL: https://issues.jboss.org/browse/TEIID-4208
> Project: Teiid
> Issue Type: Sub-task
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.0
>
>
> Increased parallelism within the plan, which includes parallelization of source queries via partitioning beyond multi-source and parallelizing the execution of union branches.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-5090) Issues with matview scripts
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-5090:
-------------------------------------
Summary: Issues with matview scripts
Key: TEIID-5090
URL: https://issues.jboss.org/browse/TEIID-5090
Project: Teiid
Issue Type: Bug
Components: Query Engine
Affects Versions: 8.12.x-6.4
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 10.0, 9.3.4
There are a couple of issues with the script logic. The first is the use of the replace function in the update criteria for updateMatView - this is not generally safe. There are also some checks for internal materialization that look for a null table in the system table, but it's actually the schema that will be null.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-4960) Problems when using External Materialized Views
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4960?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-4960:
------------------------------------
Sorry, I had that date wrong.
Thx.
> Problems when using External Materialized Views
> -----------------------------------------------
>
> Key: TEIID-4960
> URL: https://issues.jboss.org/browse/TEIID-4960
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 9.3
> Environment: * Teiid Server 9.3.0
> * Wildfly 10
> * Mysql 5.7.18
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Fix For: 10.0, 8.12.x-6.4, 9.3.1, 9.2.5, 8.12.12.6_3
>
> Attachments: server.log, teiid4960-server.log, teiid4960-vdb.xml
>
>
> The following problem occurs when, apparently, we have two views where one depend on another, and also we use External Materialization.
> The dependent view is never updated due to a "Transaction already associated with request." exception.
> Logs attached.
> The error:
> {panel:title=The exception in logs}
> 2017-06-14 13:51:51,753 WARN [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue365) Hlbq3jmDWXNl org.teiid.jdbc.TeiidSQLException: TEIID30328 Unable to evaluate mvstatus('NumberingPlan', 'numbering_plan'): TEIID30384 Error while evaluating function mvstatus
> 2017-06-14 13:52:51,803 INFO [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue368) XAtcLA8RBrLm Materialization of view NumberingPlanRaw.numbering_plan_raw started.
> 2017-06-14 13:52:51,867 ERROR [org.teiid.PROCESSOR] (Worker14_QueryProcessorQueue372) XAtcLA8RBrLm TEIID30019 Unexpected exception for request XAtcLA8RBrLm.-2514910280603581440: java.lang.AssertionError: Transaction already associated with request.
> {panel}
> The VDB:
> {code:xml}
> <model name="MnomMaterialized" type="PHYSICAL">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="query-timeout" value="600000"/>
> <source name="MnomMaterializedView" translator-name="mysql-override" connection-jndi-name="java:/mnomDs"/>
> </model>
>
> <model name="NumberingPlanCsvData">
> <source name="numberingPlanCsv-connector" translator-name="file" connection-jndi-name="java:/numberingPlanCsvDs"/>
> </model>
> <model name="NumberingPlan" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW numbering_plan (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'TRUE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY cns) as id,
> cns
> FROM (EXEC NumberingPlanCsvData.getTextFiles('NumberingPlan.csv')) AS f,
> TEXTTABLE(f.file COLUMNS cns string DELIMITER ';' SKIP 1) AS A;
>
> ]]>
> </metadata>
> </model>
>
> <model name="NumberingPlanRaw" type="VIRTUAL">
> <metadata type="DDL">
> <![CDATA[
> CREATE VIEW numbering_plan_raw (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_raw_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY network) as id,
> global_title
> FROM numbering_plan np;
>
> ]]>
> </metadata>
> </model>
> </model>
> {code}
> The Materialized table:
> {code:sql}
> CREATE TABLE status (
> VDBName VARCHAR(50) NOT NULL,
> VDBVersion VARCHAR(50) NOT NULL,
> SchemaName VARCHAR(50) NOT NULL,
> Name VARCHAR(256) NOT NULL,
> TargetSchemaName VARCHAR(50),
> TargetName VARCHAR(256) NOT NULL,
> Valid BOOLEAN NOT NULL,
> LoadState VARCHAR(25) NOT NULL,
> Cardinality BIGINT,
> Updated TIMESTAMP NOT NULL,
> LoadNumber BIGINT NOT NULL,
> NodeName varchar(25) not null,
> StaleCount BIGINT,
> PRIMARY KEY (VDBName , VDBVersion , SchemaName , Name)
> );
> CREATE TABLE numbering_plan_cache (
> id integer,
> global_title varchar(20),
> LoadNumber BIGINT,
> PRIMARY KEY(id)
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months
[JBoss JIRA] (TEIID-4960) Problems when using External Materialized Views
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4960?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4960.
-----------------------------------
Resolution: Done
This was added to 6.4 after the ER2 build. No need to reopen.
> Problems when using External Materialized Views
> -----------------------------------------------
>
> Key: TEIID-4960
> URL: https://issues.jboss.org/browse/TEIID-4960
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 9.3
> Environment: * Teiid Server 9.3.0
> * Wildfly 10
> * Mysql 5.7.18
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Fix For: 10.0, 8.12.x-6.4, 8.12.12.6_3, 9.2.5, 9.3.1
>
> Attachments: server.log, teiid4960-server.log, teiid4960-vdb.xml
>
>
> The following problem occurs when, apparently, we have two views where one depend on another, and also we use External Materialization.
> The dependent view is never updated due to a "Transaction already associated with request." exception.
> Logs attached.
> The error:
> {panel:title=The exception in logs}
> 2017-06-14 13:51:51,753 WARN [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue365) Hlbq3jmDWXNl org.teiid.jdbc.TeiidSQLException: TEIID30328 Unable to evaluate mvstatus('NumberingPlan', 'numbering_plan'): TEIID30384 Error while evaluating function mvstatus
> 2017-06-14 13:52:51,803 INFO [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue368) XAtcLA8RBrLm Materialization of view NumberingPlanRaw.numbering_plan_raw started.
> 2017-06-14 13:52:51,867 ERROR [org.teiid.PROCESSOR] (Worker14_QueryProcessorQueue372) XAtcLA8RBrLm TEIID30019 Unexpected exception for request XAtcLA8RBrLm.-2514910280603581440: java.lang.AssertionError: Transaction already associated with request.
> {panel}
> The VDB:
> {code:xml}
> <model name="MnomMaterialized" type="PHYSICAL">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="query-timeout" value="600000"/>
> <source name="MnomMaterializedView" translator-name="mysql-override" connection-jndi-name="java:/mnomDs"/>
> </model>
>
> <model name="NumberingPlanCsvData">
> <source name="numberingPlanCsv-connector" translator-name="file" connection-jndi-name="java:/numberingPlanCsvDs"/>
> </model>
> <model name="NumberingPlan" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW numbering_plan (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'TRUE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY cns) as id,
> cns
> FROM (EXEC NumberingPlanCsvData.getTextFiles('NumberingPlan.csv')) AS f,
> TEXTTABLE(f.file COLUMNS cns string DELIMITER ';' SKIP 1) AS A;
>
> ]]>
> </metadata>
> </model>
>
> <model name="NumberingPlanRaw" type="VIRTUAL">
> <metadata type="DDL">
> <![CDATA[
> CREATE VIEW numbering_plan_raw (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_raw_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY network) as id,
> global_title
> FROM numbering_plan np;
>
> ]]>
> </metadata>
> </model>
> </model>
> {code}
> The Materialized table:
> {code:sql}
> CREATE TABLE status (
> VDBName VARCHAR(50) NOT NULL,
> VDBVersion VARCHAR(50) NOT NULL,
> SchemaName VARCHAR(50) NOT NULL,
> Name VARCHAR(256) NOT NULL,
> TargetSchemaName VARCHAR(50),
> TargetName VARCHAR(256) NOT NULL,
> Valid BOOLEAN NOT NULL,
> LoadState VARCHAR(25) NOT NULL,
> Cardinality BIGINT,
> Updated TIMESTAMP NOT NULL,
> LoadNumber BIGINT NOT NULL,
> NodeName varchar(25) not null,
> StaleCount BIGINT,
> PRIMARY KEY (VDBName , VDBVersion , SchemaName , Name)
> );
> CREATE TABLE numbering_plan_cache (
> id integer,
> global_title varchar(20),
> LoadNumber BIGINT,
> PRIMARY KEY(id)
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 3 months