[JBoss JIRA] (TEIID-4398) Write a utility to convert a .VDB with Index file into -vdb.xml or DDL format
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4398?page=com.atlassian.jira.plugin... ]
Ramesh Reddy resolved TEIID-4398.
---------------------------------
Resolution: Done
Labels: CR1 (was: )
Added a utility script into "bin" directory of the installation
{code}
teiid-convert-vdb.bat/.sh
{code}
using which a .vdb or -vdb.xml file can be converted to DDL based vdb. Added blurbs about this in the docs and release notes. Will write blog about this some time after release.
> Write a utility to convert a .VDB with Index file into -vdb.xml or DDL format
> -----------------------------------------------------------------------------
>
> Key: TEIID-4398
> URL: https://issues.jboss.org/browse/TEIID-4398
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Labels: CR1
> Fix For: 9.2
>
>
> Write a command line utility and provide in "bin" directory to convert the Designer based .vdb file with index metadata into -vdb.xml file and/or newer DDL format.
> This can be used to migrate the older VDBs mach easier to newer formats without use of Designer tooling.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 11 months
[JBoss JIRA] (TEIID-4726) "java.util.ConcurrentModificationException" when deploying VDB
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4726?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4726.
-----------------------------------
Fix Version/s: 9.2
Resolution: Done
Added a catch of the concurrent modification exception rather than letting it bubble up. If this is somewhat reproducible for the customer, it would be good to use a byteman rule to capture what metadata object exactly is throwing the exception.
> "java.util.ConcurrentModificationException" when deploying VDB
> --------------------------------------------------------------
>
> Key: TEIID-4726
> URL: https://issues.jboss.org/browse/TEIID-4726
> Project: Teiid
> Issue Type: Bug
> Components: Server
> Affects Versions: 8.7.9.6_2
> Reporter: Debbie Steigner
> Assignee: Steven Hawkins
> Fix For: 9.2
>
>
> Customer is seeing "java.util.ConcurrentModificationException"[1] when deploying their VDB, the vdb is not active.
> [1]
> 06:10:47,275 ERROR [stderr] (teiid-async-threads - 3) Exception in thread "teiid-async-threads - 3" java.util.ConcurrentModificationException
> 06:10:47,276 ERROR [stderr] (teiid-async-threads - 3) at java.util.ArrayList.writeObject(ArrayList.java:746)
> 06:10:47,276 ERROR [stderr] (teiid-async-threads - 3) at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source)
> 06:10:47,276 ERROR [stderr] (teiid-async-threads - 3) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 06:10:47,276 ERROR [stderr] (teiid-async-threads - 3) at java.lang.reflect.Method.invoke(Method.java:606)
> 06:10:47,276 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
> 06:10:47,276 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
> 06:10:47,277 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,277 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,277 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> 06:10:47,288 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> 06:10:47,289 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,289 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,289 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> 06:10:47,289 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> 06:10:47,289 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,289 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,290 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> 06:10:47,290 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> 06:10:47,290 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,290 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,290 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> 06:10:47,290 ERROR [stderr] (teiid-async-threads - 3) at java.util.ArrayList.writeObject(ArrayList.java:742)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at java.lang.reflect.Method.invoke(Method.java:606)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,291 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,292 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> 06:10:47,292 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> 06:10:47,292 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,292 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,292 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> 06:10:47,292 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> 06:10:47,293 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> 06:10:47,293 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> 06:10:47,293 ERROR [stderr] (teiid-async-threads - 3) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> 06:10:47,293 ERROR [stderr] (teiid-async-threads - 3) at org.teiid.jboss.ObjectSerializer.saveAttachment(ObjectSerializer.java:77)
> 06:10:47,293 ERROR [stderr] (teiid-async-threads - 3) at org.teiid.jboss.VDBService.cacheMetadataStore(VDBService.java:468)
> 06:10:47,293 ERROR [stderr] (teiid-async-threads - 3) at org.teiid.jboss.VDBService.access$900(VDBService.java:85)
> 06:10:47,297 ERROR [stderr] (teiid-async-threads - 3) at org.teiid.jboss.VDBService$6.run(VDBService.java:405)
> 06:10:47,297 ERROR [stderr] (teiid-async-threads - 3) at org.teiid.jboss.VDBService$7.run(VDBService.java:439)
> 06:10:47,297 ERROR [stderr] (teiid-async-threads - 3) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 06:10:47,297 ERROR [stderr] (teiid-async-threads - 3) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 06:10:47,298 ERROR [stderr] (teiid-async-threads - 3) at java.lang.Thread.run(Thread.java:745)
> 06:10:47,298 ERROR [stderr] (teiid-async-threads - 3) at org.jboss.threads.JBossThread.run(JBossThread.java:122)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 11 months
[JBoss JIRA] (TEIID-4213) Cassandra translator: enable pushdown of aggregate functions
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4213?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4213:
----------------------------------
Fix Version/s: 9.3
(was: 9.2)
> Cassandra translator: enable pushdown of aggregate functions
> ------------------------------------------------------------
>
> Key: TEIID-4213
> URL: https://issues.jboss.org/browse/TEIID-4213
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.12.x
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
> Priority: Minor
> Fix For: 9.3
>
>
> Cassandra supports basic aggregate functions:
> MIN(),MAX(),SUM() and AVG()
> Currently Teiid doesn't push such queries.
> NOTE:In case of AVG() there is difference between the values returned from Teiid vs. Cassandra for an integer column. Cassandra returns only whole part of the number as integer, Teiid returns bigdecimal.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 11 months
[JBoss JIRA] (TEIID-4729) Multisource SOURCE_NAME field pushes to source
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4729?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4729.
-----------------------------------
Fix Version/s: 9.2
9.1.3
Resolution: Done
The access node was not replacing the source name deeply in the push down query, thus an inline view or subquery with the source name column would make it to the source query. A deep replacement strategy has been added.
> Multisource SOURCE_NAME field pushes to source
> ----------------------------------------------
>
> Key: TEIID-4729
> URL: https://issues.jboss.org/browse/TEIID-4729
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.6.6_3
> Reporter: Debbie Steigner
> Assignee: Steven Hawkins
> Fix For: 9.2, 9.1.3
>
>
> If a view transformation contains a union of a multi-source model and a non multisource model, because of https://issues.jboss.org/browse/TEIID-4728 the SOURCE_NAME column has to be created on the View and hence on the non-multisource model and if you don't mark that source model as multisource the SOURCE_NAME column is pushed to the source.
> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
> <vdb name="multi_emp" version="1">
> <description/>
> <property name="validationDateTime" value="Mon Oct 31 08:01:36 CDT 2016"/>
> <property name="validationVersion" value="8.12.6"/>
> <model name="multi" type="PHYSICAL" visible="true">
> <property name="multisource" value="true"/>
> <!-- optional properties -->
> <property name="multisource.columnName" value="SOURCE_NAME"/>
> <property name="multisource.addColumn" value="true"/>
>
> <source connection-jndi-name="java:/deb" name="deb" translator-name="postgresql"/>
> <source connection-jndi-name="java:/dv63" name="dv63" translator-name="postgresql"/>
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE emp (
> id integer NOT NULL OPTIONS(NAMEINSOURCE '"id"', NATIVE_TYPE 'int4', CASE_SENSITIVE 'FALSE', FIXED_LENGTH 'TRUE', SEARCHABLE 'ALL_EXCEPT_LIKE'),
> name string(2147483647) OPTIONS(NAMEINSOURCE '"name"', NATIVE_TYPE 'text'),
> SOURCE_NAME string(50),
> CONSTRAINT pk_emp PRIMARY KEY(id)
> ) OPTIONS(UPDATABLE 'true',NAMEINSOURCE '"public"."emp"')
> ]]></metadata>
> </model>
> <model name="Debbiedb" type="PHYSICAL" visible="true">
> <!-- Have to add this to make it work <property name="multisource" value="true"/> -->
> <source connection-jndi-name="java:/debbiedb" name="debbiedb" translator-name="postgresql"/>
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE mytable (
> id integer NOT NULL OPTIONS(NAMEINSOURCE '"id"', NATIVE_TYPE 'int4', CASE_SENSITIVE 'FALSE', FIXED_LENGTH 'TRUE', SEARCHABLE 'ALL_EXCEPT_LIKE'),
> name string(2147483647) OPTIONS(NAMEINSOURCE '"name"', NATIVE_TYPE 'text'),
> age integer OPTIONS(NAMEINSOURCE '"age"', NATIVE_TYPE 'int4', CASE_SENSITIVE 'FALSE', FIXED_LENGTH 'TRUE', SEARCHABLE 'ALL_EXCEPT_LIKE'),
> address string(2147483647) OPTIONS(NAMEINSOURCE '"address"', NATIVE_TYPE 'text'),
> SOURCE_NAME string(50),
> CONSTRAINT pk_id PRIMARY KEY(id)
> ) OPTIONS(UPDATABLE 'true',NAMEINSOURCE '"public"."mytable"')
> ]]></metadata>
> </model>
> <model name="Debbiev" type="VIRTUAL" visible="true">
> <property name="imports" value="Debbiedb"/>
> <property name="imports" value="multi"/>
> <metadata type="DDL"><![CDATA[
> CREATE VIEW comboview (
> id integer,
> name string(100),
> SOURCE_NAME string(50)
> )
> AS
> SELECT id,name,SOURCE_NAME FROM multi.emp UNION ALL SELECT id, name, SOURCE_NAME FROM Debbiedb.mytable
> ]]></metadata>
> </model>
> </vdb>
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 11 months
[JBoss JIRA] (TEIID-4729) Multisource SOURCE_NAME field pushes to source
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4729?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4729:
----------------------------------
Component/s: Query Engine
> Multisource SOURCE_NAME field pushes to source
> ----------------------------------------------
>
> Key: TEIID-4729
> URL: https://issues.jboss.org/browse/TEIID-4729
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.6.6_3
> Reporter: Debbie Steigner
> Assignee: Steven Hawkins
>
> If a view transformation contains a union of a multi-source model and a non multisource model, because of https://issues.jboss.org/browse/TEIID-4728 the SOURCE_NAME column has to be created on the View and hence on the non-multisource model and if you don't mark that source model as multisource the SOURCE_NAME column is pushed to the source.
> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
> <vdb name="multi_emp" version="1">
> <description/>
> <property name="validationDateTime" value="Mon Oct 31 08:01:36 CDT 2016"/>
> <property name="validationVersion" value="8.12.6"/>
> <model name="multi" type="PHYSICAL" visible="true">
> <property name="multisource" value="true"/>
> <!-- optional properties -->
> <property name="multisource.columnName" value="SOURCE_NAME"/>
> <property name="multisource.addColumn" value="true"/>
>
> <source connection-jndi-name="java:/deb" name="deb" translator-name="postgresql"/>
> <source connection-jndi-name="java:/dv63" name="dv63" translator-name="postgresql"/>
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE emp (
> id integer NOT NULL OPTIONS(NAMEINSOURCE '"id"', NATIVE_TYPE 'int4', CASE_SENSITIVE 'FALSE', FIXED_LENGTH 'TRUE', SEARCHABLE 'ALL_EXCEPT_LIKE'),
> name string(2147483647) OPTIONS(NAMEINSOURCE '"name"', NATIVE_TYPE 'text'),
> SOURCE_NAME string(50),
> CONSTRAINT pk_emp PRIMARY KEY(id)
> ) OPTIONS(UPDATABLE 'true',NAMEINSOURCE '"public"."emp"')
> ]]></metadata>
> </model>
> <model name="Debbiedb" type="PHYSICAL" visible="true">
> <!-- Have to add this to make it work <property name="multisource" value="true"/> -->
> <source connection-jndi-name="java:/debbiedb" name="debbiedb" translator-name="postgresql"/>
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE mytable (
> id integer NOT NULL OPTIONS(NAMEINSOURCE '"id"', NATIVE_TYPE 'int4', CASE_SENSITIVE 'FALSE', FIXED_LENGTH 'TRUE', SEARCHABLE 'ALL_EXCEPT_LIKE'),
> name string(2147483647) OPTIONS(NAMEINSOURCE '"name"', NATIVE_TYPE 'text'),
> age integer OPTIONS(NAMEINSOURCE '"age"', NATIVE_TYPE 'int4', CASE_SENSITIVE 'FALSE', FIXED_LENGTH 'TRUE', SEARCHABLE 'ALL_EXCEPT_LIKE'),
> address string(2147483647) OPTIONS(NAMEINSOURCE '"address"', NATIVE_TYPE 'text'),
> SOURCE_NAME string(50),
> CONSTRAINT pk_id PRIMARY KEY(id)
> ) OPTIONS(UPDATABLE 'true',NAMEINSOURCE '"public"."mytable"')
> ]]></metadata>
> </model>
> <model name="Debbiev" type="VIRTUAL" visible="true">
> <property name="imports" value="Debbiedb"/>
> <property name="imports" value="multi"/>
> <metadata type="DDL"><![CDATA[
> CREATE VIEW comboview (
> id integer,
> name string(100),
> SOURCE_NAME string(50)
> )
> AS
> SELECT id,name,SOURCE_NAME FROM multi.emp UNION ALL SELECT id, name, SOURCE_NAME FROM Debbiedb.mytable
> ]]></metadata>
> </model>
> </vdb>
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 11 months
[JBoss JIRA] (TEIID-4730) Problem with Import VDB with Materialized View (external)
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4730?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4730:
-------------------------------------
how does your 'CountryServiceListVDB' VDB look like? Can u also attach that?
> Problem with Import VDB with Materialized View (external)
> ---------------------------------------------------------
>
> Key: TEIID-4730
> URL: https://issues.jboss.org/browse/TEIID-4730
> Project: Teiid
> Issue Type: Bug
> Components: Common
> Affects Versions: 9.1.2
> Environment: * Teiid 9.1.2
> * CentOs 7
> * WildFly 10
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Attachments: externalMaterializationOrderProblem-vdb.xml
>
>
> Having defined a VDB which contains a Model that has External Materialization, when importing this VDB into another that also uses External Materialization, Teiid Server is searching in the wrong place for the view status.
> Example:
> ..
> <import-vdb name="CountryServiceListVDB" version="1" import-data-policies="true"/>
> ...
> <model name="MyView" type="VIRTUAL">
> ...
> SELECT cns,
> country_code
> FROM NumberingPlan.numbering_plan;
> ....
> </metadata>
> </model>
> *NumberingPlan.numbering_plan is defined in the imported VDB, and is also Materialized.*
> *Warning in Logs: *
> 17:51:45,161 WARN [org.teiid.MATVIEWS] (Worker8_QueryProcessorQueue10653) n9M6de3y5xHM org.teiid.jdbc.TeiidSQLException: TEIID30328 Unable to evaluate (SELECT mvstatus('NumberingPlan', 'numbering_plan', Valid, LoadState, 'THROW_EXCEPTION') FROM (SELECT 1) AS x LEFT OUTER JOIN NumberingPlanMaterialized.status ON VDBName = 'ExternalMaterializationOrderProblem' AND VDBVersion = '1' AND SchemaName = 'NumberingPlan' AND Name = 'numbering_plan' LIMIT 2): TEIID30328 Unable to evaluate mvstatus('NumberingPlan', 'numbering_plan', Valid, LoadState, 'THROW_EXCEPTION'): TEIID30384 Error while evaluating function mvstatus
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 11 months