[JBoss JIRA] (TEIIDSB-170) Automate materialization to JDG
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-170?page=com.atlassian.jira.plug... ]
Ramesh Reddy commented on TEIIDSB-170:
--------------------------------------
> They simply are providing a materialization target (or similar) unique name, which is used to drive the base names of the caches created.
I am saying any of these require further user input that that takes away the usability of it as user expected to additional input.
> I'm not looking forward to maintaining two parsers. I'd prefer most of the other solutions to this one.
IMO, one does not need to write a full parser, just enough for the commands, and this is the best option for complete automation. It is not like language features keep extending, these 3 different statements that we are interested in IMO are much static in nature. The problem of cache names is not alone, the issue also exists for maven pom.xml generation, if we want foolproof one this is it.
> Automate materialization to JDG
> -------------------------------
>
> Key: TEIIDSB-170
> URL: https://issues.redhat.com/browse/TEIIDSB-170
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: OpenShift
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
> Original Estimate: 1 week
> Time Spent: 1 week, 3 days, 2 hours
> Remaining Estimate: 0 minutes
>
> Create an internal materialization replacement needs that is turnkey materialization to JDG (little to no user setup required)
> - the operator may create the infinispan cluster if needed
> - the status table and internal representation of the materialization target would be setup automatically
> For the user this would be as simple marking a view as materialized and then it would be populated in jdg upon deployment. They would not have any concerns with cache naming, status tables, etc.
> For simplicity the initial version would make a similar assumption to the current internal logic - it is for only a specific vdb. If the vdb cr is modified, then it's expected that the cache would be recreated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-170) Automate materialization to JDG
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-170?page=com.atlassian.jira.plug... ]
Steven Hawkins edited comment on TEIIDSB-170 at 5/11/20 1:22 PM:
-----------------------------------------------------------------
> Actually, this does not work, the scanning we are looking at is done before the maven code-gen is run, so most of the techniques fall into the user's lap to have the knowledge to enter these, thus failing the automation step.
I don't follow you on this. They simply are providing a materialization target (or similar) unique name, which is used to drive the base names of the caches created.
> I have started looking at yacc passed parser in GO to see if I can write a parser for only the DDL statements that we need currently. Steven Hawkins can you give examples of inline comments in SQL for these as you mentioned above? Looking through SQL grammar file in Teiid, I find only one place like
I'm not looking forward to maintaining two parsers. I'd prefer most of the other solutions to this one.
> I was not sure any others exist?
End of line comments
{code} -- I'm a comment {code}
And inline/multi-line comments are supported - I even added comment nesting to further complicate things.
{code}
/* I'm a big
comment
*/
{code}
{code}
/* I'm a big
comment
/* with a nested comment */
*/
{code}
was (Author: shawkins):
> Actually, this does not work, the scanning we are looking at is done before the maven code-gen is run, so most of the techniques fall into the user's lap to have the knowledge to enter these, thus failing the automation step.
I don't follow you on this. They simply are providing a materialization target (or similar) unique name, which is used to drive the base names of the caches created.
> I have started looking at yacc passed parser in GO to see if I can write a parser for only the DDL statements that we need currently. Steven Hawkins can you give examples of inline comments in SQL for these as you mentioned above? Looking through SQL grammar file in Teiid, I find only one place like
I'm not looking forward to maintaining two parsers. I'd prefer most of the other solutions to this one.
> I was not sure any others exist?
Multi-line comments are also supported - I even added comment nesting to further complicate things.
> Automate materialization to JDG
> -------------------------------
>
> Key: TEIIDSB-170
> URL: https://issues.redhat.com/browse/TEIIDSB-170
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: OpenShift
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
> Original Estimate: 1 week
> Time Spent: 1 week, 3 days, 2 hours
> Remaining Estimate: 0 minutes
>
> Create an internal materialization replacement needs that is turnkey materialization to JDG (little to no user setup required)
> - the operator may create the infinispan cluster if needed
> - the status table and internal representation of the materialization target would be setup automatically
> For the user this would be as simple marking a view as materialized and then it would be populated in jdg upon deployment. They would not have any concerns with cache naming, status tables, etc.
> For simplicity the initial version would make a similar assumption to the current internal logic - it is for only a specific vdb. If the vdb cr is modified, then it's expected that the cache would be recreated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-170) Automate materialization to JDG
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-170?page=com.atlassian.jira.plug... ]
Steven Hawkins commented on TEIIDSB-170:
----------------------------------------
> Actually, this does not work, the scanning we are looking at is done before the maven code-gen is run, so most of the techniques fall into the user's lap to have the knowledge to enter these, thus failing the automation step.
I don't follow you on this. They simply are providing a materialization target (or similar) unique name, which is used to drive the base names of the caches created.
> I have started looking at yacc passed parser in GO to see if I can write a parser for only the DDL statements that we need currently. Steven Hawkins can you give examples of inline comments in SQL for these as you mentioned above? Looking through SQL grammar file in Teiid, I find only one place like
I'm not looking forward to maintaining two parsers. I'd prefer most of the other solutions to this one.
> I was not sure any others exist?
Multi-line comments are also supported - I even added comment nesting to further complicate things.
> Automate materialization to JDG
> -------------------------------
>
> Key: TEIIDSB-170
> URL: https://issues.redhat.com/browse/TEIIDSB-170
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: OpenShift
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
> Original Estimate: 1 week
> Time Spent: 1 week, 3 days, 2 hours
> Remaining Estimate: 0 minutes
>
> Create an internal materialization replacement needs that is turnkey materialization to JDG (little to no user setup required)
> - the operator may create the infinispan cluster if needed
> - the status table and internal representation of the materialization target would be setup automatically
> For the user this would be as simple marking a view as materialized and then it would be populated in jdg upon deployment. They would not have any concerns with cache naming, status tables, etc.
> For simplicity the initial version would make a similar assumption to the current internal logic - it is for only a specific vdb. If the vdb cr is modified, then it's expected that the cache would be recreated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5939) connection pool not destroyed after datasource deletion without server reload(java)
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5939?focusedWorklogId=12451005&pag... ]
Steven Hawkins logged work on TEIID-5939:
-----------------------------------------
Author: Steven Hawkins
Created on: 11/May/20 1:02 PM
Start Date: 11/May/20 1:02 PM
Worklog Time Spent: 2 hours
Issue Time Tracking
-------------------
Remaining Estimate: 1 hour (was: 3 hours)
Time Spent: 2 hours
Worklog Id: (was: 12451005)
> connection pool not destroyed after datasource deletion without server reload(java)
> -----------------------------------------------------------------------------------
>
> Key: TEIID-5939
> URL: https://issues.redhat.com/browse/TEIID-5939
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
> Original Estimate: 3 hours
> Time Spent: 2 hours
> Remaining Estimate: 1 hour
>
> Hi Team,
> Connection pool is not removed after deletion of datasource using java code.
> Teiid(Jboss) server reload required to reflect the deletion of connection pool from actual database(we are using mysql as database).
> Until server reloads, connections in deleted datasource's connection pool remain in sleep mode.
> *Pre-requisite*
> * Connection is created with the mysql database using teiid
> * Datasource is created in teiid.
> * Delete the above created connection through java using code i.e by using AdminApiClientAccessor.
> * That removed the datasource from teiid.
> * Through below code I am able to delete the datasource.
> *Java code*
> {code:java}
> import org.teiid.adminapi.Admin;
> ..
> private Admin admin; //assigned object to admin;
> ..
> ..
> public void deleteDataSource(String datasourceName){
> try {
> admin.deleteDataSource(datasourceName);
> } catch (Exception e) {
> logger.error(e);
> }
> }
> {code}
> datasourceName = name of the datasource created earlier.
> *standalone-teiid.xml*
> </xa-datasource>
> ..
> ..
> <xa-pool>
> <min-pool-size>3</min-pool-size>
> <max-pool-size>10</max-pool-size>
> <flush-strategy>IdleConnections</flush-strategy>
> </xa-pool>
> <validation>
> <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"/>
> <background-validation>true</background-validation>
> <exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"/>
> </validation>
> </xa-datasource>
> *Issues :*
> * Even though datasource removed, connection pool is not destroyed
> * Able to see connection threads available on Mysql admin console using the command
>
> {code:java}
> show processlist;
> {code}
> * Changes reflect only when the Teiid (jboss) server is reloaded.
> * Once server reloads connections in connection pool is closed/removed from the database.
> * 'Too many connection' issue facing on mysql server due to multiple connection threads remains in sleep mode even though datasource are already removed.
> Please let us know :
> * Is there any method to remove connection pool from actual database.
> * Is server reload mandatory.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5939) connection pool not destroyed after datasource deletion without server reload(java)
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5939?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5939:
----------------------------------
Original Estimate: 3 hours
Remaining Estimate: 3 hours
Story Points: 0.5
Sprint: DV Sprint 63
> connection pool not destroyed after datasource deletion without server reload(java)
> -----------------------------------------------------------------------------------
>
> Key: TEIID-5939
> URL: https://issues.redhat.com/browse/TEIID-5939
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
> Original Estimate: 3 hours
> Remaining Estimate: 3 hours
>
> Hi Team,
> Connection pool is not removed after deletion of datasource using java code.
> Teiid(Jboss) server reload required to reflect the deletion of connection pool from actual database(we are using mysql as database).
> Until server reloads, connections in deleted datasource's connection pool remain in sleep mode.
> *Pre-requisite*
> * Connection is created with the mysql database using teiid
> * Datasource is created in teiid.
> * Delete the above created connection through java using code i.e by using AdminApiClientAccessor.
> * That removed the datasource from teiid.
> * Through below code I am able to delete the datasource.
> *Java code*
> {code:java}
> import org.teiid.adminapi.Admin;
> ..
> private Admin admin; //assigned object to admin;
> ..
> ..
> public void deleteDataSource(String datasourceName){
> try {
> admin.deleteDataSource(datasourceName);
> } catch (Exception e) {
> logger.error(e);
> }
> }
> {code}
> datasourceName = name of the datasource created earlier.
> *standalone-teiid.xml*
> </xa-datasource>
> ..
> ..
> <xa-pool>
> <min-pool-size>3</min-pool-size>
> <max-pool-size>10</max-pool-size>
> <flush-strategy>IdleConnections</flush-strategy>
> </xa-pool>
> <validation>
> <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"/>
> <background-validation>true</background-validation>
> <exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"/>
> </validation>
> </xa-datasource>
> *Issues :*
> * Even though datasource removed, connection pool is not destroyed
> * Able to see connection threads available on Mysql admin console using the command
>
> {code:java}
> show processlist;
> {code}
> * Changes reflect only when the Teiid (jboss) server is reloaded.
> * Once server reloads connections in connection pool is closed/removed from the database.
> * 'Too many connection' issue facing on mysql server due to multiple connection threads remains in sleep mode even though datasource are already removed.
> Please let us know :
> * Is there any method to remove connection pool from actual database.
> * Is server reload mandatory.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5939) connection pool not destroyed after datasource deletion without server reload(java)
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5939?page=com.atlassian.jira.plugi... ]
Steven Hawkins resolved TEIID-5939.
-----------------------------------
Resolution: Done
Updated the adminimpl to set the appropriate header to immediately delete the datasource.
> connection pool not destroyed after datasource deletion without server reload(java)
> -----------------------------------------------------------------------------------
>
> Key: TEIID-5939
> URL: https://issues.redhat.com/browse/TEIID-5939
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
>
> Hi Team,
> Connection pool is not removed after deletion of datasource using java code.
> Teiid(Jboss) server reload required to reflect the deletion of connection pool from actual database(we are using mysql as database).
> Until server reloads, connections in deleted datasource's connection pool remain in sleep mode.
> *Pre-requisite*
> * Connection is created with the mysql database using teiid
> * Datasource is created in teiid.
> * Delete the above created connection through java using code i.e by using AdminApiClientAccessor.
> * That removed the datasource from teiid.
> * Through below code I am able to delete the datasource.
> *Java code*
> {code:java}
> import org.teiid.adminapi.Admin;
> ..
> private Admin admin; //assigned object to admin;
> ..
> ..
> public void deleteDataSource(String datasourceName){
> try {
> admin.deleteDataSource(datasourceName);
> } catch (Exception e) {
> logger.error(e);
> }
> }
> {code}
> datasourceName = name of the datasource created earlier.
> *standalone-teiid.xml*
> </xa-datasource>
> ..
> ..
> <xa-pool>
> <min-pool-size>3</min-pool-size>
> <max-pool-size>10</max-pool-size>
> <flush-strategy>IdleConnections</flush-strategy>
> </xa-pool>
> <validation>
> <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"/>
> <background-validation>true</background-validation>
> <exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"/>
> </validation>
> </xa-datasource>
> *Issues :*
> * Even though datasource removed, connection pool is not destroyed
> * Able to see connection threads available on Mysql admin console using the command
>
> {code:java}
> show processlist;
> {code}
> * Changes reflect only when the Teiid (jboss) server is reloaded.
> * Once server reloads connections in connection pool is closed/removed from the database.
> * 'Too many connection' issue facing on mysql server due to multiple connection threads remains in sleep mode even though datasource are already removed.
> Please let us know :
> * Is there any method to remove connection pool from actual database.
> * Is server reload mandatory.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5939) connection pool not destroyed after datasource deletion without server reload(java)
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5939?page=com.atlassian.jira.plugi... ]
Work on TEIID-5939 started by Steven Hawkins.
---------------------------------------------
> connection pool not destroyed after datasource deletion without server reload(java)
> -----------------------------------------------------------------------------------
>
> Key: TEIID-5939
> URL: https://issues.redhat.com/browse/TEIID-5939
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
>
> Hi Team,
> Connection pool is not removed after deletion of datasource using java code.
> Teiid(Jboss) server reload required to reflect the deletion of connection pool from actual database(we are using mysql as database).
> Until server reloads, connections in deleted datasource's connection pool remain in sleep mode.
> *Pre-requisite*
> * Connection is created with the mysql database using teiid
> * Datasource is created in teiid.
> * Delete the above created connection through java using code i.e by using AdminApiClientAccessor.
> * That removed the datasource from teiid.
> * Through below code I am able to delete the datasource.
> *Java code*
> {code:java}
> import org.teiid.adminapi.Admin;
> ..
> private Admin admin; //assigned object to admin;
> ..
> ..
> public void deleteDataSource(String datasourceName){
> try {
> admin.deleteDataSource(datasourceName);
> } catch (Exception e) {
> logger.error(e);
> }
> }
> {code}
> datasourceName = name of the datasource created earlier.
> *standalone-teiid.xml*
> </xa-datasource>
> ..
> ..
> <xa-pool>
> <min-pool-size>3</min-pool-size>
> <max-pool-size>10</max-pool-size>
> <flush-strategy>IdleConnections</flush-strategy>
> </xa-pool>
> <validation>
> <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"/>
> <background-validation>true</background-validation>
> <exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"/>
> </validation>
> </xa-datasource>
> *Issues :*
> * Even though datasource removed, connection pool is not destroyed
> * Able to see connection threads available on Mysql admin console using the command
>
> {code:java}
> show processlist;
> {code}
> * Changes reflect only when the Teiid (jboss) server is reloaded.
> * Once server reloads connections in connection pool is closed/removed from the database.
> * 'Too many connection' issue facing on mysql server due to multiple connection threads remains in sleep mode even though datasource are already removed.
> Please let us know :
> * Is there any method to remove connection pool from actual database.
> * Is server reload mandatory.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-170) Automate materialization to JDG
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-170?page=com.atlassian.jira.plug... ]
Ramesh Reddy commented on TEIIDSB-170:
--------------------------------------
>> by reusing the MATERIALIZATION_TARGET property
>This will work actually, this directly represents the cache name. I am ok with this. This is the most simple one.
Actually, this does not work, the scanning we are looking at is done before the maven code-gen is run, so most of the techniques fall into the user's lap to have the knowledge to enter these, thus failing the automation step.
> but yes a full parser of these multiple statements is probably a better way to do this.
I have started looking at yacc passed parser in GO to see if I can write a parser for *only* the DDL statements that we need currently. [~shawkins] can you give examples of inline comments in SQL for these as you mentioned above? Looking through SQL grammar file in Teiid, I find only one place like
{code}
CREATE VIEW {Name} ( columns ...) OPTIONS (...) AS /* comment */ SELECT ...
{code}
I was not sure any others exist?
> Automate materialization to JDG
> -------------------------------
>
> Key: TEIIDSB-170
> URL: https://issues.redhat.com/browse/TEIIDSB-170
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: OpenShift
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
> Original Estimate: 1 week
> Time Spent: 1 week, 3 days, 2 hours
> Remaining Estimate: 0 minutes
>
> Create an internal materialization replacement needs that is turnkey materialization to JDG (little to no user setup required)
> - the operator may create the infinispan cluster if needed
> - the status table and internal representation of the materialization target would be setup automatically
> For the user this would be as simple marking a view as materialized and then it would be populated in jdg upon deployment. They would not have any concerns with cache naming, status tables, etc.
> For simplicity the initial version would make a similar assumption to the current internal logic - it is for only a specific vdb. If the vdb cr is modified, then it's expected that the cache would be recreated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5736) General odata function mapping
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5736?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5736:
----------------------------------
Fix Version/s: (was: 14.0)
> General odata function mapping
> ------------------------------
>
> Key: TEIID-5736
> URL: https://issues.redhat.com/browse/TEIID-5736
> Project: Teiid
> Issue Type: Quality Risk
> Components: OData
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 14.1
>
>
> We are currently mapping procedures and functions defined by ddl to odata functions/actions. The latter is due that the function is represented in our system metadata as a procedure. We need to decide if all functions should be mapped. There's also the notion of a composable function that could be helpful.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months