[JBoss JIRA] (TEIID-5657) VDB Migration Tool
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5657?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-5657:
------------------------------------
The description indicates a jar is created, just doesn't mention which jar. Also, I don't a migrate jar or a repo of such. Where can I find the jar?
> VDB Migration Tool
> ------------------
>
> Key: TEIID-5657
> URL: https://issues.jboss.org/browse/TEIID-5657
> Project: Teiid
> Issue Type: Feature Request
> Components: Documentation, Quick Starts, Tooling
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Priority: Major
>
> A migration tool will be provided to convert customer vdbs into just ddl. More than likely this will be just a single ddl file - which will probably not be acceptable to customers with large vdbs.
> We have this utility already available as just a main method in one of our jars. It is anticipated that a maven repo will be distribution mechanism and minimal documentation will be provided.
> There should be optional validation available for the vdb ddl at build time. This breaks down into 3 parts:
> 1. static syntactic validation, potentially even fully resolving if all metadata is present. This makes sure that basic typos will be caught.
> 2. providing hard errors for things that are completely removed - an error for usage of soap or function models for example.
> 3. providing errors or warning for features that are not yet available - vdb imports, sources that aren't yet supported etc.
> To avoid introducing a new plugin or plugging into fmp it's been suggested that the validation could be run via a generated unit test, which ties into another task which is booster, base project, or last resort an arche type that scaffolds the developer project.
> This could possibly be included in the quickstarts as a means to facilitate its use.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIIDSB-23?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIIDSB-23:
---------------------------------------
> I was trying to use what we maybe using in future with syndesis so that there for a single monitoring solution.
Yes, I'm just questioning dataintegration as the type.
> Only thing I want to understand is if there are multiple pods with "datavirtualization" label, all of them can be discovered and monitored by a single Prometheus system.
That is correct. The config actually allows for both integration and datavirtualization - but the actual metrics collection is just Teiid.
> I was not saying to dynamically change what is deployed as the previous example
I'm not talking about dynamic per se. There is the f-m-p approach of fabric8-includes, and there is a template approach where you can reference the same base image, but use a new deployment config that sets all of the relevant labels/annotations and provides a configmap with the prometheus exporter config.
> I came across this article https://www.callicoder.com/spring-boot-actuator-metrics-monitoring-dashbo... where we could use micrometer with Teiid app and see if we can expose some of readymade metrics.
There are no shortage of approaches. There's jolokia, the jmx_exporter, and micrometer. Micrometer is effectively moving the image prometheus config yaml to annotations and the meter filter properties file - that does have some additional features like local histograms and other meters that would otherwise be computed on prometheus.
As everything that we're interested in is already in jmx and the jmx_exporter including jvm metrics micrometer's not doing much for us yet. It's also possible to have different collection jobs so you could collect whatever from the jmx_exporter and from micrometer, etc.
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.jboss.org/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: Q119
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIID-5657) VDB Migration Tool
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5657?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-5657:
------------------------------------
Ok, I can do that.
> VDB Migration Tool
> ------------------
>
> Key: TEIID-5657
> URL: https://issues.jboss.org/browse/TEIID-5657
> Project: Teiid
> Issue Type: Feature Request
> Components: Documentation, Quick Starts, Tooling
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Priority: Major
>
> A migration tool will be provided to convert customer vdbs into just ddl. More than likely this will be just a single ddl file - which will probably not be acceptable to customers with large vdbs.
> We have this utility already available as just a main method in one of our jars. It is anticipated that a maven repo will be distribution mechanism and minimal documentation will be provided.
> There should be optional validation available for the vdb ddl at build time. This breaks down into 3 parts:
> 1. static syntactic validation, potentially even fully resolving if all metadata is present. This makes sure that basic typos will be caught.
> 2. providing hard errors for things that are completely removed - an error for usage of soap or function models for example.
> 3. providing errors or warning for features that are not yet available - vdb imports, sources that aren't yet supported etc.
> To avoid introducing a new plugin or plugging into fmp it's been suggested that the validation could be run via a generated unit test, which ties into another task which is booster, base project, or last resort an arche type that scaffolds the developer project.
> This could possibly be included in the quickstarts as a means to facilitate its use.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIID-5657) VDB Migration Tool
by Ramesh Reddy (Jira)
[ https://issues.jboss.org/browse/TEIID-5657?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5657:
-------------------------------------
[~van.halbert] if you want you can try it out with few .vdb/.xml files and see how the user experience is, or bugs in generating the DDL files.
> VDB Migration Tool
> ------------------
>
> Key: TEIID-5657
> URL: https://issues.jboss.org/browse/TEIID-5657
> Project: Teiid
> Issue Type: Feature Request
> Components: Documentation, Quick Starts, Tooling
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Priority: Major
>
> A migration tool will be provided to convert customer vdbs into just ddl. More than likely this will be just a single ddl file - which will probably not be acceptable to customers with large vdbs.
> We have this utility already available as just a main method in one of our jars. It is anticipated that a maven repo will be distribution mechanism and minimal documentation will be provided.
> There should be optional validation available for the vdb ddl at build time. This breaks down into 3 parts:
> 1. static syntactic validation, potentially even fully resolving if all metadata is present. This makes sure that basic typos will be caught.
> 2. providing hard errors for things that are completely removed - an error for usage of soap or function models for example.
> 3. providing errors or warning for features that are not yet available - vdb imports, sources that aren't yet supported etc.
> To avoid introducing a new plugin or plugging into fmp it's been suggested that the validation could be run via a generated unit test, which ties into another task which is booster, base project, or last resort an arche type that scaffolds the developer project.
> This could possibly be included in the quickstarts as a means to facilitate its use.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Ramesh Reddy (Jira)
[ https://issues.jboss.org/browse/TEIIDSB-23?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIIDSB-23:
-------------------------------------
>used the term datavirtualization, rather than dataintegration as integration is already a primary term for syndesis.
I was trying to use what we maybe using in future with syndesis so that there for a single monitoring solution.
>discovery of the pod is based upon the label (not annotation) syndesis.io/type=datavirtualization - this can of course be changed, but needs to be done on >both the deployment config for our image and in the prometheus config
I think this is fine, I see that is approach syndesis used. Only thing I want to understand is if there are multiple pods with "datavirtualization" label, all of them can be discovered and monitored by a single Prometheus system.
> it is possible to separate this into just the prometheus example
I was not saying to dynamically change what is deployed as the previous example. I was only thinking in terms of instructions and README with duplication of code/configuration from previous example. I will provide updates to that.
I came across this article https://www.callicoder.com/spring-boot-actuator-metrics-monitoring-dashbo... where we could use micrometer with Teiid app and see if we can expose some of readymade metrics.
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.jboss.org/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: Q119
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIID-5643) When querying a larger dataset, teiid answers with 504 Gateway Time-out and throws exception
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5643?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5643:
---------------------------------------
> Ok, to what level should I decrease the memory and which parameters are relevant for this? Probably one of the following?
Yes, you could try -Xmx256m to create a very constrained instance.
Another thing to try is to take odata out of the picture - can you run the representative query over jdbc without issue? And in general how long does it take?
> When querying a larger dataset, teiid answers with 504 Gateway Time-out and throws exception
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-5643
> URL: https://issues.jboss.org/browse/TEIID-5643
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 11.2.1
> Reporter: Christoph John
> Assignee: Steven Hawkins
> Priority: Major
> Attachments: Heap Dump analysis tool.pdf
>
>
> Please note, I am using Teiid 11.2.0 and not 11.2.1 as specified above in this ticket. I was not able to select a version 11.2.0.
> The issue: I tried to query a table with a larger amount of records (about 600.000 records in the table). Teiid responds with
> 504 Gateway Time-out. The server didn't respond in time.
> If Teiid tries to respond with 600.000 entries to such a request, it is reasonable that it runs out of memory or other weird things happen. However, I would expect that in such a case Teiid just answers with a fixed amount of records, say for example 1000, and provides a kind of curser to enable subsequent requests to retrieve more data.
> I am currently trying to find a workaround for the issue, as the issue blocks my work. Is there maybe a kind of configuration option that I have to set explicitly for such a kind of behavior I previously sketched?
> Further note: When I send such a kind of select * request on the table to Teiid, I am also not able for a certain time interval, about a minute, to get a responds to different queries which select a single element on the mentioned table. Such a request then also results in a 504 Gateway Time-out.
> Attached error log:
> 2019-02-01 22:33:36,126 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (Worker2_QueryProcessorQueue5) eayVFuq9y7OY IJ000407: No lazy enlistment available for my_nutri_diary
> 2019-02-01 22:33:36,135 INFO [org.teiid.CONNECTOR] (Worker2_QueryProcessorQueue5) eayVFuq9y7OY MySQLExecutionFactory Commit=true;DatabaseProductName=MySQL;DatabaseProductVersion=8.0.13;DriverMajorVersion=8;DriverMajorVersion=0;DriverName=MySQL Connector/J;DriverVersion=mysql-connector-java-8.0.13 (Revision: 66459e9d39c8fd09767992bc592acd2053279be6);IsolationLevel=2
> 2019-02-01 22:36:56,959 WARN [org.teiid.CONNECTOR] (Worker2_QueryProcessorQueue5) eayVFuq9y7OY Connector worker process failed for atomic-request=eayVFuq9y7OY.0.0.0: org.teiid.translator.jdbc.JDBCExecutionException: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.`idCode` AS c_0, g_0.`lc` AS c_1, g_0.`product_name` AS c_2, g_0.`origins` AS c_3, g_0.`brands` AS c_4, g_0.`quantity` AS c_5, g_0.`nova_group` AS c_6, g_0.`nutrition_grade_fr` AS c_7, g_0.`ingredients_text_with_allergens` AS c_8, g_0.`energy_100g` AS c_9, g_0.`carbohydrates_100g` AS c_10, g_0.`sugars_100g` AS c_11, g_0.`proteins_100g` AS c_12, g_0.`fat_100g` AS c_13, g_0.`saturated_fat_100g` AS c_14, g_0.`saturated_fat_modifier` AS c_15, g_0.`salt_100g` AS c_16, g_0.`sodium_100g` AS c_17 FROM `FDBProducts` AS g_0 ORDER BY c_0]
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:127)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:393)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:229)
> at com.sun.proxy.$Proxy51.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:61)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:278)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:115)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:206)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.sql.SQLException: Java heap space
> at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
> at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
> at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
> at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:974)
> at com.mysql.cj.jdbc.ClientPreparedStatement.executeQuery(ClientPreparedStatement.java:1024)
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:504)
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:119)
> ... 18 more
> 2019-02-01 22:36:56,977 WARN [org.teiid.PROCESSOR] (default task-4) eayVFuq9y7OY TEIID30020 Processing exception for request eayVFuq9y7OY.0 'TEIID30504 my_nutri_diary: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.`idCode` AS c_0, g_0.`lc` AS c_1, g_0.`product_name` AS c_2, g_0.`origins` AS c_3, g_0.`brands` AS c_4, g_0.`quantity` AS c_5, g_0.`nova_group` AS c_6, g_0.`nutrition_grade_fr` AS c_7, g_0.`ingredients_text_with_allergens` AS c_8, g_0.`energy_100g` AS c_9, g_0.`carbohydrates_100g` AS c_10, g_0.`sugars_100g` AS c_11, g_0.`proteins_100g` AS c_12, g_0.`fat_100g` AS c_13, g_0.`saturated_fat_100g` AS c_14, g_0.`saturated_fat_modifier` AS c_15, g_0.`salt_100g` AS c_16, g_0.`sodium_100g` AS c_17 FROM `FDBProducts` AS g_0 ORDER BY c_0]'. Originally TeiidProcessingException 'Java heap space' SQLError.java:129. Enable more detailed logging to see the entire stacktrace.
> 2019-02-01 22:36:56,987 WARN [org.teiid.PROCESSOR] (default task-4) TEIID16053 Unable to process odata request due to: TEIID30504 my_nutri_diary: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.`idCode` AS c_0, g_0.`lc` AS c_1, g_0.`product_name` AS c_2, g_0.`origins` AS c_3, g_0.`brands` AS c_4, g_0.`quantity` AS c_5, g_0.`nova_group` AS c_6, g_0.`nutrition_grade_fr` AS c_7, g_0.`ingredients_text_with_allergens` AS c_8, g_0.`energy_100g` AS c_9, g_0.`carbohydrates_100g` AS c_10, g_0.`sugars_100g` AS c_11, g_0.`proteins_100g` AS c_12, g_0.`fat_100g` AS c_13, g_0.`saturated_fat_100g` AS c_14, g_0.`saturated_fat_modifier` AS c_15, g_0.`salt_100g` AS c_16, g_0.`sodium_100g` AS c_17 FROM `FDBProducts` AS g_0 ORDER BY c_0] Increase the log level to see the entire stack trace.
> 2019-02-01 22:36:57,213 WARN [org.teiid.CONNECTOR] (Worker2_QueryProcessorQueue6) NdaDihMR2Vlt Connector worker process failed for atomic-request=NdaDihMR2Vlt.0.0.1: org.teiid.translator.jdbc.JDBCExecutionException: 0 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.`idCode` AS c_0, g_0.`lc` AS c_1, g_0.`product_name` AS c_2, g_0.`origins` AS c_3, g_0.`brands` AS c_4, g_0.`quantity` AS c_5, g_0.`nova_group` AS c_6, g_0.`nutrition_grade_fr` AS c_7, g_0.`ingredients_text_with_allergens` AS c_8, g_0.`energy_100g` AS c_9, g_0.`carbohydrates_100g` AS c_10, g_0.`sugars_100g` AS c_11, g_0.`proteins_100g` AS c_12, g_0.`fat_100g` AS c_13, g_0.`saturated_fat_100g` AS c_14, g_0.`saturated_fat_modifier` AS c_15, g_0.`salt_100g` AS c_16, g_0.`sodium_100g` AS c_17 FROM `FDBProducts` AS g_0 ORDER BY c_0]
> at org.teiid.translator.jdbc.JDBCQueryExecution.execute(JDBCQueryExecution.java:127)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:393)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:229)
> at com.sun.proxy.$Proxy51.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:61)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:278)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:115)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:206)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIIDSB-23?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIIDSB-23:
---------------------------------------
For convenience moving the pr comment here:
* used the term datavirtualization, rather than dataintegration as integration is already a primary term for syndesis.
* discovery of the pod is based upon the label (not annotation) syndesis.io/type=datavirtualization - this can of course be changed, but needs to be done on both the deployment config for our image and in the prometheus config
* metric_relabel_configs is a bit tricky. what syndesis has is redundant - a single keep causes everything that does not match to be dropped. keep also makes it hard to merge keep rules as you need to merge the regex. Note that theirs uses the context type as a prefix - which does not of course match our type labels (currently cache and runtime). We'll likely need to come up with better strategy here if we are going to share a prometheus instance.
* We are following syndesis conventions. The metrics have java style names, org.teiid.xxx, but . is automatically replaced with _ in the actual name - likely to make for easier regex matching.
* it is possible to separate this into just the prometheus example if it's based upon a new deployment config file that references the image built by rdbms_example and overlays the export configuration via a configmap, rather than using fabric8 include
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.jboss.org/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: Q119
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIID-5657) VDB Migration Tool
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5657?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-5657:
------------------------------------
[~rareddy] reassigning to you, as you to ownership of the product jira. Let me know if there's anything I can do.
> VDB Migration Tool
> ------------------
>
> Key: TEIID-5657
> URL: https://issues.jboss.org/browse/TEIID-5657
> Project: Teiid
> Issue Type: Feature Request
> Components: Documentation, Quick Starts, Tooling
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Priority: Major
>
> A migration tool will be provided to convert customer vdbs into just ddl. More than likely this will be just a single ddl file - which will probably not be acceptable to customers with large vdbs.
> We have this utility already available as just a main method in one of our jars. It is anticipated that a maven repo will be distribution mechanism and minimal documentation will be provided.
> There should be optional validation available for the vdb ddl at build time. This breaks down into 3 parts:
> 1. static syntactic validation, potentially even fully resolving if all metadata is present. This makes sure that basic typos will be caught.
> 2. providing hard errors for things that are completely removed - an error for usage of soap or function models for example.
> 3. providing errors or warning for features that are not yet available - vdb imports, sources that aren't yet supported etc.
> To avoid introducing a new plugin or plugging into fmp it's been suggested that the validation could be run via a generated unit test, which ties into another task which is booster, base project, or last resort an arche type that scaffolds the developer project.
> This could possibly be included in the quickstarts as a means to facilitate its use.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIID-5657) VDB Migration Tool
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5657?page=com.atlassian.jira.plugin... ]
Van Halbert reassigned TEIID-5657:
----------------------------------
Assignee: Ramesh Reddy (was: Van Halbert)
> VDB Migration Tool
> ------------------
>
> Key: TEIID-5657
> URL: https://issues.jboss.org/browse/TEIID-5657
> Project: Teiid
> Issue Type: Feature Request
> Components: Documentation, Quick Starts, Tooling
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Priority: Major
>
> A migration tool will be provided to convert customer vdbs into just ddl. More than likely this will be just a single ddl file - which will probably not be acceptable to customers with large vdbs.
> We have this utility already available as just a main method in one of our jars. It is anticipated that a maven repo will be distribution mechanism and minimal documentation will be provided.
> There should be optional validation available for the vdb ddl at build time. This breaks down into 3 parts:
> 1. static syntactic validation, potentially even fully resolving if all metadata is present. This makes sure that basic typos will be caught.
> 2. providing hard errors for things that are completely removed - an error for usage of soap or function models for example.
> 3. providing errors or warning for features that are not yet available - vdb imports, sources that aren't yet supported etc.
> To avoid introducing a new plugin or plugging into fmp it's been suggested that the validation could be run via a generated unit test, which ties into another task which is booster, base project, or last resort an arche type that scaffolds the developer project.
> This could possibly be included in the quickstarts as a means to facilitate its use.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month
[JBoss JIRA] (TEIID-5636) Planning error. Could not find symbol disiz__1.DischargeTime
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5636?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5636:
---------------------------------------
The issue is with RulePlanOuterJoins.planLeftOuterJoinAssociativityBeforePlanning. A workaround is to use the preserve hint:
{code}
SELECT "BAAQREP"."AQANCD" AS "BAAQREP_AQBUTX"
FROM /*+ preserve */ ("SAMPLEModel_fb4V"."DB2ADMIN"."BAAFREP" "BAAFREP"
LEFT JOIN "BInvModel_4eGO"."Omega"."dbo"."ClientTypeL_cpF_240119" "ClientTypeL_cpF_240119"
ON ("BAAFREP"."AFAJST" = "ClientTypeL_cpF_240119"."ClientTypeIndicator")
LEFT JOIN "SAMPLEModel_fb4V"."DB2ADMIN"."BAAGREP" "BAAGREP"
ON ("BAAFREP"."AFAINB" = "BAAGREP"."AGAINB")
LEFT JOIN "SAMPLEModel_fb4V"."DB2ADMIN"."BAAIREP" "BAAIREP"
ON ("BAAGREP"."AGAVNB" = "BAAIREP"."AIAVNB")
LEFT JOIN "SAMPLEModel_fb4V"."DB2ADMIN"."BAAOREP" "BAAOREP"
ON ("BAAIREP"."AIAKCD" = "BAAOREP"."AOAKCD")
) LEFT JOIN "SAMPLEModel_fb4V"."DB2ADMIN"."BAAQREP" "BAAQREP"
ON ("BAAIREP"."AIANCD" = "BAAQREP"."AQANCD")
LEFT JOIN "SAMPLEModel_fb4V"."DB2ADMIN"."BAAPREP" "BAAPREP"
ON ("BAAIREP"."AIALCD" = "BAAPREP"."APALCD")
LEFT JOIN "BInvModel_4eGO"."Omega"."dbo"."Identificat_joa_170119" "Identificat_joa_170119"
ON ("BAAIREP"."AIAKST" = "Identificat_joa_170119"."RNMIdenTypeValue")
LEFT JOIN "OmegaModel_qgWs"."Omega"."Cpte"."CPARTFCP" "CPARTFCP"
ON (("BAAFREP"."AFAINB" = Convert("CPARTFCP"."_COMPTE", FLOAT)))
LIMIT 0, 10
{code}
I'll provide a fix today.
> Planning error. Could not find symbol disiz__1.DischargeTime
> ------------------------------------------------------------
>
> Key: TEIID-5636
> URL: https://issues.jboss.org/browse/TEIID-5636
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 10.3.4
> Reporter: Bram Gadeyne
> Assignee: Steven Hawkins
> Priority: Critical
> Attachments: planning_error.txt
>
>
> Hi,
> We're running into this exception. It happens when adding "LEFT OUTER JOIN izisprod.P_DischargeData". I unfortunitaly have no smaller example. We use this table all the time without any previous errors.
> 2019-01-31 15:32:36,438 ERROR [org.teiid.PROCESSOR] (Worker172_QueryProcessorQueue57002) JesgvCmnKfP5 TEIID30019 Unexpected exception for request JesgvCmnKfP5.187: org.teiid.core.TeiidRuntimeException: Planning error. Could not find symbol: disiz__1.DischargeTime
> at org.teiid.query.processor.relational.RelationalNode.getProjectionIndexes(RelationalNode.java:363)
> at org.teiid.query.processor.relational.JoinNode.initialize(JoinNode.java:129)
> at org.teiid.query.processor.relational.RelationalPlan.connectExternal(RelationalPlan.java:92)
> at org.teiid.query.processor.relational.RelationalPlan.connectExternal(RelationalPlan.java:98)
> at org.teiid.query.processor.relational.RelationalPlan.connectExternal(RelationalPlan.java:98)
> at org.teiid.query.processor.relational.RelationalPlan.initialize(RelationalPlan.java:87)
> at org.teiid.query.processor.QueryProcessor.init(QueryProcessor.java:223)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:135)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:111)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:480)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:350)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:47)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:276)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:277)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:115)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:206)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years, 1 month