[JBoss JIRA] (TEIID-4960) Problems when using External Materialized Views
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4960?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4960:
---------------------------------------
> It is evident from the error message that this not currently supported
It is expected to work. I'd like to confirm first if this is the same as TEIID-4961 - do you see the same details for the mvstatus failures?
> Problems when using External Materialized Views
> -----------------------------------------------
>
> Key: TEIID-4960
> URL: https://issues.jboss.org/browse/TEIID-4960
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 9.3
> Environment: * Teiid Server 9.3.0
> * Wildfly 10
> * Mysql 5.7.18
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Attachments: server.log
>
>
> The following problem occurs when, apparently, we have two views where one depend on another, and also we use External Materialization.
> The dependent view is never updated due to a "Transaction already associated with request." exception.
> Logs attached.
> The error:
> {panel:title=The exception in logs}
> 2017-06-14 13:51:51,753 WARN [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue365) Hlbq3jmDWXNl org.teiid.jdbc.TeiidSQLException: TEIID30328 Unable to evaluate mvstatus('NumberingPlan', 'numbering_plan'): TEIID30384 Error while evaluating function mvstatus
> 2017-06-14 13:52:51,803 INFO [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue368) XAtcLA8RBrLm Materialization of view NumberingPlanRaw.numbering_plan_raw started.
> 2017-06-14 13:52:51,867 ERROR [org.teiid.PROCESSOR] (Worker14_QueryProcessorQueue372) XAtcLA8RBrLm TEIID30019 Unexpected exception for request XAtcLA8RBrLm.-2514910280603581440: java.lang.AssertionError: Transaction already associated with request.
> {panel}
> The VDB:
> {code:xml}
> <model name="MnomMaterialized" type="PHYSICAL">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="query-timeout" value="600000"/>
> <source name="MnomMaterializedView" translator-name="mysql-override" connection-jndi-name="java:/mnomDs"/>
> </model>
>
> <model name="NumberingPlanCsvData">
> <source name="numberingPlanCsv-connector" translator-name="file" connection-jndi-name="java:/numberingPlanCsvDs"/>
> </model>
> <model name="NumberingPlan" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW numbering_plan (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'TRUE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY cns) as id,
> cns
> FROM (EXEC NumberingPlanCsvData.getTextFiles('NumberingPlan.csv')) AS f,
> TEXTTABLE(f.file COLUMNS cns string DELIMITER ';' SKIP 1) AS A;
>
> ]]>
> </metadata>
> </model>
>
> <model name="NumberingPlanRaw" type="VIRTUAL">
> <metadata type="DDL">
> <![CDATA[
> CREATE VIEW numbering_plan_raw (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_raw_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY network) as id,
> global_title
> FROM numbering_plan np;
>
> ]]>
> </metadata>
> </model>
> </model>
> {code}
> The Materialized table:
> {code:sql}
> CREATE TABLE status (
> VDBName VARCHAR(50) NOT NULL,
> VDBVersion VARCHAR(50) NOT NULL,
> SchemaName VARCHAR(50) NOT NULL,
> Name VARCHAR(256) NOT NULL,
> TargetSchemaName VARCHAR(50),
> TargetName VARCHAR(256) NOT NULL,
> Valid BOOLEAN NOT NULL,
> LoadState VARCHAR(25) NOT NULL,
> Cardinality BIGINT,
> Updated TIMESTAMP NOT NULL,
> LoadNumber BIGINT NOT NULL,
> NodeName varchar(25) not null,
> StaleCount BIGINT,
> PRIMARY KEY (VDBName , VDBVersion , SchemaName , Name)
> );
> CREATE TABLE numbering_plan_cache (
> id integer,
> global_title varchar(20),
> LoadNumber BIGINT,
> PRIMARY KEY(id)
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (TEIID-4961) External Materialized View With State Loaded but 0 Cardinality
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4961?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4961:
---------------------------------------
What materializations is SfrBaseSource.sfr_base_source dependent upon?
> External Materialized View With State Loaded but 0 Cardinality
> --------------------------------------------------------------
>
> Key: TEIID-4961
> URL: https://issues.jboss.org/browse/TEIID-4961
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 9.3
> Environment: * Teiid 9.3.0
> * MySQL 5.7.18
> * Wildfly 10
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Attachments: server.log
>
>
> *Scenario*: Having several views that depend on others (ex: V1-> V2->V3, where V3 depends on V2 and V2 depends on V1), all with external materialization.
> Although all views LoadState is *LOADED *even after failures due to the dependent ones being loaded, in the end some views have zero (0) of cardinality, when there are actually records. Running the queries by hand return records.
> Log attached.
> Example:
> {panel}
> 2017-06-14 17:14:39,521 INFO [org.teiid.MATVIEWS] (Worker63_QueryProcessorQueue19799) WDpoyHmXlkvS Materialization of view VodafoneNlBaseSource.vodafone_nl_base_source completed. Rows updated = 0 Load Number = 1
> {panel}
> After forcing the view update:
> {panel}
> 2017-06-14 17:23:38,042 INFO [org.teiid.MATVIEWS] (Worker65_QueryProcessorQueue27597) +IbRaNRtrZ1K Materialization of view VodafoneNlBaseSource.vodafone_nl_base_source completed. Rows updated = 1816 Load Number = 2
> {panel}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (TEIID-4964) AWS S3 Connector
by Chandra Akkinepalli (JIRA)
Chandra Akkinepalli created TEIID-4964:
------------------------------------------
Summary: AWS S3 Connector
Key: TEIID-4964
URL: https://issues.jboss.org/browse/TEIID-4964
Project: Teiid
Issue Type: Feature Request
Reporter: Chandra Akkinepalli
Assignee: Steven Hawkins
I would like to request a feature for a new connector to access files in AWS S3 via JDV and be able to use SQL to query them.
It would be great to have the ability to use AWS KMS for accessing the SSE-KMS encrypted objects on AWS S3
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (TEIID-4307) Infinispan DSL Translator: Can't execute update when foreign table name differs from POJO's simple name
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-4307?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-4307:
------------------------------------------------
Jan Stastny <jstastny(a)redhat.com> changed the Status of [bug 1351093|https://bugzilla.redhat.com/show_bug.cgi?id=1351093] from ON_QA to VERIFIED
> Infinispan DSL Translator: Can't execute update when foreign table name differs from POJO's simple name
> -------------------------------------------------------------------------------------------------------
>
> Key: TEIID-4307
> URL: https://issues.jboss.org/browse/TEIID-4307
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Affects Versions: 8.12.5
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Fix For: 9.1
>
>
> When using Infinispan DSL translator to query data in a jdg instance using POJO, proto file and marshaller, then there's an issue when the foreign table name in a vdb doesn't match simple name of the POJO class.
> Resource adapter:
> {code:xml}
> <resource-adapter id="jdg-mat2">
> <module slot="main" id="org.jboss.teiid.resource-adapter.infinispan.dsl"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.infinispan.dsl.InfinispanManagedConnectionFactory" jndi-name="java:/jdg-mat2-ds" enabled="true" pool-name="jdg-mat2">
> <config-property name="CacheTypeMap">
> rhel7openjdk1_8_localjdg-mat:org.jboss.qe.jdg.pojo.another.CustomerReport2;customerId
> </config-property>
> <config-property name="ProtobufDefinitionFile">
> /org/jboss/qe/jdg/remote/protobuf/another/CustomerReport2.proto
> </config-property>
> <config-property name="Module">
> org.jboss.qe.jdg.pojos
> </config-property>
> <config-property name="MessageDescriptor">
> org.jboss.qe.jdg.remote.protobuf.another.CustomerReport2
> </config-property>
> <config-property name="RemoteServerList">
> dvqe01.mw.lab.eng.bos.redhat.com:11322
> </config-property>
> <config-property name="MessageMarshallers">
> org.jboss.qe.jdg.pojo.another.CustomerReport2:org.jboss.qe.jdg.remote.marshaller.AnotherCustomerReportMarshaller
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> Source model:
> {code:xml}
> <source name="jdgSource2" translator-name="infinispan-cache-dsl"
> connection-jndi-name="java:/jdg-mat2-ds" />
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE Customer_Report2(
> customerId integer not null OPTIONS (SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> CONSTRAINT PK_ID PRIMARY KEY(customerId)
> ) OPTIONS (NAMEINSOURCE 'rhel7openjdk1_8_localjdg-mat', UPDATABLE TRUE);]]>
> </metadata>
> </model>
> {code}
> Then performing update query (INSERT, UPDATE, DELETE) causes problem.
> Example query:
> {code:sql}
> INSERT INTO jdgSource2.Customer_Report2 VALUES (18);
> {code}
> results in:
> {code:plain}
> 10:00:04,657 WARN [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue4) Connector worker process failed for atomic-request=2v5rhJOyKrpA.2.0.2: org.teiid.translator.TranslatorException: TEIID21005 Registered Class not found for table Customer_Report2, check table name matches a registered class
> {code}
> If there's a requirement for equality between POJO class simple name and modelled table name, it should be clearly stated in documentation as this is very important limitation. Moreover Java classes and SQL tables tend to have different naming conventions.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (TEIID-4961) External Materialized View With State Loaded but 0 Cardinality
by Pedro Inácio (JIRA)
[ https://issues.jboss.org/browse/TEIID-4961?page=com.atlassian.jira.plugin... ]
Pedro Inácio commented on TEIID-4961:
-------------------------------------
The MATVIEW_ONERROR_ACTION is set to WAIT, and the result is the one reported in this issue.
> External Materialized View With State Loaded but 0 Cardinality
> --------------------------------------------------------------
>
> Key: TEIID-4961
> URL: https://issues.jboss.org/browse/TEIID-4961
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 9.3
> Environment: * Teiid 9.3.0
> * MySQL 5.7.18
> * Wildfly 10
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Attachments: server.log
>
>
> *Scenario*: Having several views that depend on others (ex: V1-> V2->V3, where V3 depends on V2 and V2 depends on V1), all with external materialization.
> Although all views LoadState is *LOADED *even after failures due to the dependent ones being loaded, in the end some views have zero (0) of cardinality, when there are actually records. Running the queries by hand return records.
> Log attached.
> Example:
> {panel}
> 2017-06-14 17:14:39,521 INFO [org.teiid.MATVIEWS] (Worker63_QueryProcessorQueue19799) WDpoyHmXlkvS Materialization of view VodafoneNlBaseSource.vodafone_nl_base_source completed. Rows updated = 0 Load Number = 1
> {panel}
> After forcing the view update:
> {panel}
> 2017-06-14 17:23:38,042 INFO [org.teiid.MATVIEWS] (Worker65_QueryProcessorQueue27597) +IbRaNRtrZ1K Materialization of view VodafoneNlBaseSource.vodafone_nl_base_source completed. Rows updated = 1816 Load Number = 2
> {panel}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (TEIID-4960) Problems when using External Materialized Views
by Pedro Inácio (JIRA)
[ https://issues.jboss.org/browse/TEIID-4960?page=com.atlassian.jira.plugin... ]
Pedro Inácio commented on TEIID-4960:
-------------------------------------
The problem is that currently I've around 30 views, from which 20 depend on one another. Some views take almost 12 minutes to became available. In total model takes around 35 minutes to load. I'm using external materialization exactly to overcome the load time delay since these views can be frequently updated, and each one individually must be always available to consultation.
> Problems when using External Materialized Views
> -----------------------------------------------
>
> Key: TEIID-4960
> URL: https://issues.jboss.org/browse/TEIID-4960
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 9.3
> Environment: * Teiid Server 9.3.0
> * Wildfly 10
> * Mysql 5.7.18
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Attachments: server.log
>
>
> The following problem occurs when, apparently, we have two views where one depend on another, and also we use External Materialization.
> The dependent view is never updated due to a "Transaction already associated with request." exception.
> Logs attached.
> The error:
> {panel:title=The exception in logs}
> 2017-06-14 13:51:51,753 WARN [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue365) Hlbq3jmDWXNl org.teiid.jdbc.TeiidSQLException: TEIID30328 Unable to evaluate mvstatus('NumberingPlan', 'numbering_plan'): TEIID30384 Error while evaluating function mvstatus
> 2017-06-14 13:52:51,803 INFO [org.teiid.MATVIEWS] (Worker14_QueryProcessorQueue368) XAtcLA8RBrLm Materialization of view NumberingPlanRaw.numbering_plan_raw started.
> 2017-06-14 13:52:51,867 ERROR [org.teiid.PROCESSOR] (Worker14_QueryProcessorQueue372) XAtcLA8RBrLm TEIID30019 Unexpected exception for request XAtcLA8RBrLm.-2514910280603581440: java.lang.AssertionError: Transaction already associated with request.
> {panel}
> The VDB:
> {code:xml}
> <model name="MnomMaterialized" type="PHYSICAL">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="query-timeout" value="600000"/>
> <source name="MnomMaterializedView" translator-name="mysql-override" connection-jndi-name="java:/mnomDs"/>
> </model>
>
> <model name="NumberingPlanCsvData">
> <source name="numberingPlanCsv-connector" translator-name="file" connection-jndi-name="java:/numberingPlanCsvDs"/>
> </model>
> <model name="NumberingPlan" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW numbering_plan (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'TRUE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY cns) as id,
> cns
> FROM (EXEC NumberingPlanCsvData.getTextFiles('NumberingPlan.csv')) AS f,
> TEXTTABLE(f.file COLUMNS cns string DELIMITER ';' SKIP 1) AS A;
>
> ]]>
> </metadata>
> </model>
>
> <model name="NumberingPlanRaw" type="VIRTUAL">
> <metadata type="DDL">
> <![CDATA[
> CREATE VIEW numbering_plan_raw (
> id integer PRIMARY KEY,
> global_title varchar(20)
> )
> OPTIONS(
> MATERIALIZED 'TRUE',
> UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'MnomMaterialized.numbering_plan_raw_cache',
> "teiid_rel:MATVIEW_TTL" 86400000,
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOADNUMBER_COLUMN" 'LoadNumber',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'MnomMaterialized.status'
> )
> AS
> SELECT ROW_NUMBER() OVER (ORDER BY network) as id,
> global_title
> FROM numbering_plan np;
>
> ]]>
> </metadata>
> </model>
> </model>
> {code}
> The Materialized table:
> {code:sql}
> CREATE TABLE status (
> VDBName VARCHAR(50) NOT NULL,
> VDBVersion VARCHAR(50) NOT NULL,
> SchemaName VARCHAR(50) NOT NULL,
> Name VARCHAR(256) NOT NULL,
> TargetSchemaName VARCHAR(50),
> TargetName VARCHAR(256) NOT NULL,
> Valid BOOLEAN NOT NULL,
> LoadState VARCHAR(25) NOT NULL,
> Cardinality BIGINT,
> Updated TIMESTAMP NOT NULL,
> LoadNumber BIGINT NOT NULL,
> NodeName varchar(25) not null,
> StaleCount BIGINT,
> PRIMARY KEY (VDBName , VDBVersion , SchemaName , Name)
> );
> CREATE TABLE numbering_plan_cache (
> id integer,
> global_title varchar(20),
> LoadNumber BIGINT,
> PRIMARY KEY(id)
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (TEIID-4963) JDBC LocalProfile should not bind with jboss-modules
by Kylin Soong (JIRA)
Kylin Soong created TEIID-4963:
----------------------------------
Summary: JDBC LocalProfile should not bind with jboss-modules
Key: TEIID-4963
URL: https://issues.jboss.org/browse/TEIID-4963
Project: Teiid
Issue Type: Enhancement
Components: JDBC Driver
Affects Versions: 10.0
Reporter: Kylin Soong
Assignee: Kylin Soong
Fix For: 10.0
JDBC LocalProfile seems strictly bind with jboss-modules
{code}
class ModuleHelper {
static ServerConnection createFromModule(Properties info)
throws ConnectionException, TeiidException {
ClassLoader tccl = Thread.currentThread().getContextClassLoader();
try {
ModuleLoader callerModuleLoader = Module.getCallerModuleLoader();
if (callerModuleLoader == null) {
throw new ConnectionException(JDBCPlugin.Event.TEIID20033, null, JDBCPlugin.Util.gs(JDBCPlugin.Event.TEIID20033));
}
final Module module = callerModuleLoader.loadModule(ModuleIdentifier.create("org.jboss.teiid")); //$NON-NLS-1$
Thread.currentThread().setContextClassLoader(module.getClassLoader());
return (ServerConnection)ReflectionHelper.create("org.teiid.transport.LocalServerConnection", Arrays.asList(info, PropertiesUtils.getBooleanProperty(info, LocalProfile.USE_CALLING_THREAD, true)), Thread.currentThread().getContextClassLoader()); //$NON-NLS-1$
} catch (ModuleLoadException e) {
throw new ConnectionException(JDBCPlugin.Event.TEIID20008, e, JDBCPlugin.Util.gs(JDBCPlugin.Event.TEIID20008));
} finally {
Thread.currentThread().setContextClassLoader(tccl);
}
}
}
{code}
These code may have risk when invoked by no-jboss environment, eg, the below is the stacktrace of spring boot invoke these code
{code}
Caused by: org.teiid.net.ConnectionException: TEIID20033 Cannot use a local connection as no Teiid instance was detected.
at org.teiid.jdbc.ModuleHelper.createFromModule(ModuleHelper.java:45) ~[teiid-client-10.0.0.Alpha1-SNAPSHOT.jar:10.0.0.Alpha1-SNAPSHOT]
at org.teiid.jdbc.LocalProfile.createServerConnection(LocalProfile.java:56) ~[teiid-client-10.0.0.Alpha1-SNAPSHOT.jar:10.0.0.Alpha1-SNAPSHOT]
at org.teiid.jdbc.LocalProfile.connect(LocalProfile.java:46) ~[teiid-client-10.0.0.Alpha1-SNAPSHOT.jar:10.0.0.Alpha1-SNAPSHOT]
... 45 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years