June 2020 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-5965) Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters

by Steven Hawkins (Jira)

[ https://issues.redhat.com/browse/TEIID-5965?page=com.atlassian.jira.plugi... ] Steven Hawkins updated TEIID-5965: ---------------------------------- Fix Version/s: 15.0 Original Estimate: 5 hours Remaining Estimate: 5 hours Story Points: 1 In total you are requesting delimiter, header and skip be parameterizable. That is possible. I think the initial rationale for static values was for easier usage / analysis in Teiid Designer. > Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters > ------------------------------------------------------------------------------------------------------------------- > > Key: TEIID-5965 > URL: https://issues.redhat.com/browse/TEIID-5965 > Project: Teiid > Issue Type: Enhancement > Components: Query Engine > Affects Versions: 13.1 > Reporter: Dmitrii Pogorelov > Assignee: Steven Hawkins > Priority: Major > Fix For: 15.0 > > Original Estimate: 5 hours > Remaining Estimate: 5 hours > > In the specified example, the delimiter is TAB. Many web APIs allow customization of the delimiter character, and in order to provide a generic parser, sometimes it is way easier to define a delimiter as a variable, and not need to create a long nested structure with IF-THEN-ELSE-IF-ELSE constructs: > {code:sql} > Select * From TextTable ( > 'c1 c2 > 1 2' > Columns > c1 integer, > c2 integer > Delimiter E'\t' > Header 1 > )x; > {code} > Imagine that based on setup on API side (out of our control) the content can be delivered via tab or semicolon, e.g. > {code} > c1;c2 > 1;2 > {code} > or > {code} > c1 c2 > 1 2 > {code} > Let's save this response into a variable and see the code, which we will need to write depending on the setup: > {code:sql} > Begin > ... > If (delimiter = 'tab') > Begin > Select * From TextTable ( > apiResponse > Columns > c1 integer, > c2 integer > Delimiter E'\t' > Header 1 > )x; > End > Else If (delimiter = 'tab') > Begin > Select * From TextTable ( > apiResponse > Columns > c1 integer, > c2 integer > Delimiter ';' > Header 1 > )x; > End > End > {code} > The if-else block is constantly growing, especially if we want to customize quote, escape, delimiter, and row delimiter. > Thus it would be great if we could make these values configurable. In this case, we could end up with this expected code, which is more readable and more easily customizable: > {code:sql} > Begin > ... > Declare string delimiter = E'\t'; > Select * From TextTable ( > apiResponse > Columns > c1 integer, > c2 integer > Delimiter delimiter > Header 1 > )x; > End > {code} > With a bit of tweaking and certain assumptions, leading in the trust level, we can even read the first line and try to auto-detect the delimiter automatically (e.g. by counting tabs, commas, and semicolons in the first line). > Please, could you be so kind as to make HEADER and SKIP values customizable (rather than hardcoded numbers)? -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIIDSB-207) Generate a Teiid Spring Boot project from given YAML based CR file

by Ramesh Reddy (Jira)

Ramesh Reddy created TEIIDSB-207: ------------------------------------ Summary: Generate a Teiid Spring Boot project from given YAML based CR file Key: TEIIDSB-207 URL: https://issues.redhat.com/browse/TEIIDSB-207 Project: Teiid Spring Boot Issue Type: Feature Request Components: tools Reporter: Ramesh Reddy Assignee: Ramesh Reddy Extend the "vdb-codegen" plugin to generate a full Teiid Spring Boot project based on the VDB defined in YAML CR. Currently the plugin already generates the `Application.java` and `DataSource.java` from a given DDL file, but when YAML file is supplied it should also generate the `pom.xml` and `application.properties` to be self-standing full TSB project as in this example https://github.com/teiid/teiid-spring-boot/tree/master/samples/vdb The intent behind is to use this for the local building of the project for testing with VSCode Plugin. Based on how this works out, could also be used in Operator work flow. -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5967) Disallow the usage of handler in create fdw

by Steven Hawkins (Jira)

[ https://issues.redhat.com/browse/TEIID-5967?focusedWorklogId=12451216&pag... ] Steven Hawkins logged work on TEIID-5967: ----------------------------------------- Author: Steven Hawkins Created on: 03/Jun/20 1:28 PM Start Date: 03/Jun/20 1:28 PM Worklog Time Spent: 30 minutes Issue Time Tracking ------------------- Remaining Estimate: 0 minutes (was: 30 minutes) Time Spent: 30 minutes Worklog Id: (was: 12451216) > Disallow the usage of handler in create fdw > ------------------------------------------- > > Key: TEIID-5967 > URL: https://issues.redhat.com/browse/TEIID-5967 > Project: Teiid > Issue Type: Quality Risk > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 15.0 > > Original Estimate: 30 minutes > Time Spent: 30 minutes > Remaining Estimate: 0 minutes > > To reserve it for future use we should not be using handler as a synonym to type in create foreign data wrapper. -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5967) Disallow the usage of handler in create fdw

by Steven Hawkins (Jira)

[ https://issues.redhat.com/browse/TEIID-5967?page=com.atlassian.jira.plugi... ] Steven Hawkins resolved TEIID-5967. ----------------------------------- Resolution: Done Updated the parser and added a release note. > Disallow the usage of handler in create fdw > ------------------------------------------- > > Key: TEIID-5967 > URL: https://issues.redhat.com/browse/TEIID-5967 > Project: Teiid > Issue Type: Quality Risk > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 15.0 > > Original Estimate: 30 minutes > Remaining Estimate: 30 minutes > > To reserve it for future use we should not be using handler as a synonym to type in create foreign data wrapper. -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5967) Disallow the usage of handler in create fdw

by Steven Hawkins (Jira)

Steven Hawkins created TEIID-5967: ------------------------------------- Summary: Disallow the usage of handler in create fdw Key: TEIID-5967 URL: https://issues.redhat.com/browse/TEIID-5967 Project: Teiid Issue Type: Quality Risk Components: Query Engine Reporter: Steven Hawkins Assignee: Steven Hawkins Fix For: 15.0 To reserve it for future use we should not be using handler as a synonym to type in create foreign data wrapper. -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5963) Teiid imported all schemas from PG

by Steven Hawkins (Jira)

[ https://issues.redhat.com/browse/TEIID-5963?page=com.atlassian.jira.plugi... ] Steven Hawkins commented on TEIID-5963: --------------------------------------- > Why if we make hibernate connection to DB we can just disable loading of all metadata, but not in teiid? The simple answer is that hibernate is not using the getTypeInfo method in your usage scenario. And again this is not all metadata, just type information for which the standard JDBC DatabaseMetaData interface does not provide an ability to limit per schema. > Second, why not also try disabling the retrieving type info as suggested to see if it improves things? That won't work. The type fetching happens regardless of that option. > Can we set properties (or make beans config) in spring boot app to change this behavior? Currently there is no solution via just properties. As Ramesh say the only option is to capture the metadata you want to avoid the metadata fetching at startup time. > Teiid imported all schemas from PG > ---------------------------------- > > Key: TEIID-5963 > URL: https://issues.redhat.com/browse/TEIID-5963 > Project: Teiid > Issue Type: Bug > Components: JDBC Connector > Affects Versions: 14.0 > Reporter: Renat Eskenin > Assignee: Steven Hawkins > Priority: Major > > When we started teiid spring boot app, we have long start time. -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5962) Teiid locks many threads when Salesforce is disconnected

by Ramesh Reddy (Jira)

[ https://issues.redhat.com/browse/TEIID-5962?page=com.atlassian.jira.plugi... ] Ramesh Reddy commented on TEIID-5962: ------------------------------------- There are two properties defined in this https://github.com/teiid/teiid-spring-boot/blob/master/data/salesforce/sr... > How to set this properties in teiid spring boot application? May be bean config or properties? in the `application.properties` as shown in this example https://github.com/teiid/teiid-spring-boot/blob/master/samples/salesforce... you can set {code} spring.teiid.data.salesforce.requestTimeout=5000 spring.teiid.data.salesforce.connectTimeout=5000 {code} requestTimeout is per request after the connection is made, connectTimeout is during the connection. BTW, note here that underneath we use Salesforce libraries, so the logic to enforce is done inside it. Let us know if that works? > Teiid locks many threads when Salesforce is disconnected > -------------------------------------------------------- > > Key: TEIID-5962 > URL: https://issues.redhat.com/browse/TEIID-5962 > Project: Teiid > Issue Type: Bug > Components: Salesforce Connector > Affects Versions: 12.3.1 > Reporter: Renat Eskenin > Assignee: Steven Hawkins > Priority: Major > Attachments: Снимок экрана от 2020-05-31 11-09-10.png > > > We had two incidents in our experimental service with teiid spring boot app. When Salesforce is down to 20min and service with teiid trying to call SQL from Salesforce we have many threads in tomcat server (in spring boot app). Then when 200 threads is activated service do not reply to requests because is "max threads" in tomcat of service. > Why teiid do not send response as "Salesforce is down - error" or connection timeout? > Micro schema: > HTTP Requests to REST api -> Teiid spring boot app -> {broken 20min}Salesforce -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5963) Teiid imported all schemas from PG

by Ramesh Reddy (Jira)

[ https://issues.redhat.com/browse/TEIID-5963?page=com.atlassian.jira.plugi... ] Ramesh Reddy commented on TEIID-5963: ------------------------------------- > Can we set properties (or make beans config) in spring boot app to change this behavior? You can disable loading metadata from data source, but for that you need to remove {code} IMPORT FOREIGN SCHEMA manual_loads FROM SERVER pgserver INTO schpostgresql OPTIONS("importer.useFullSchemaName" 'false', "importer.tableTypes" 'TABLE,VIEW'); {code} and then replace it with all the table definitions that represent your PG database statically. I have not seen your confirmation from you for [~shawkins] question that is in fact type info which is taking long (am I reading wrong?). Second, why not also try disabling the retrieving type info as suggested to see if it improves things? for that you need to set {code} importer.widenUnsignedTypes = false {code} on the vdb. Also can you update the version to `1.5.1` instead of `1.5.0` as I made couple enhancements with property handling. > Teiid imported all schemas from PG > ---------------------------------- > > Key: TEIID-5963 > URL: https://issues.redhat.com/browse/TEIID-5963 > Project: Teiid > Issue Type: Bug > Components: JDBC Connector > Affects Versions: 14.0 > Reporter: Renat Eskenin > Assignee: Steven Hawkins > Priority: Major > > When we started teiid spring boot app, we have long start time. -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5962) Teiid locks many threads when Salesforce is disconnected

by Steven Hawkins (Jira)

[ https://issues.redhat.com/browse/TEIID-5962?page=com.atlassian.jira.plugi... ] Steven Hawkins commented on TEIID-5962: --------------------------------------- > How to set this properties in teiid spring boot application? May be bean config or properties? Are you running Teiid Spring Boot? Generally you'd create a TEIIDSB issue for that instead. On the Teiid Spring Boot side yes there are typically configuration beans: https://github.com/teiid/teiid-spring-boot/blob/master/data/salesforce/sr... In this case the supporting logic came from Teiid, so the property set is the same. > Teiid locks many threads when Salesforce is disconnected > -------------------------------------------------------- > > Key: TEIID-5962 > URL: https://issues.redhat.com/browse/TEIID-5962 > Project: Teiid > Issue Type: Bug > Components: Salesforce Connector > Affects Versions: 12.3.1 > Reporter: Renat Eskenin > Assignee: Steven Hawkins > Priority: Major > Attachments: Снимок экрана от 2020-05-31 11-09-10.png > > > We had two incidents in our experimental service with teiid spring boot app. When Salesforce is down to 20min and service with teiid trying to call SQL from Salesforce we have many threads in tomcat server (in spring boot app). Then when 200 threads is activated service do not reply to requests because is "max threads" in tomcat of service. > Why teiid do not send response as "Salesforce is down - error" or connection timeout? > Micro schema: > HTTP Requests to REST api -> Teiid spring boot app -> {broken 20min}Salesforce -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5962) Teiid locks many threads when Salesforce is disconnected

by Renat Eskenin (Jira)

[ https://issues.redhat.com/browse/TEIID-5962?page=com.atlassian.jira.plugi... ] Renat Eskenin commented on TEIID-5962: -------------------------------------- How to set this properties in teiid spring boot application? May be bean config or properties? > Teiid locks many threads when Salesforce is disconnected > -------------------------------------------------------- > > Key: TEIID-5962 > URL: https://issues.redhat.com/browse/TEIID-5962 > Project: Teiid > Issue Type: Bug > Components: Salesforce Connector > Affects Versions: 12.3.1 > Reporter: Renat Eskenin > Assignee: Steven Hawkins > Priority: Major > Attachments: Снимок экрана от 2020-05-31 11-09-10.png > > > We had two incidents in our experimental service with teiid spring boot app. When Salesforce is down to 20min and service with teiid trying to call SQL from Salesforce we have many threads in tomcat server (in spring boot app). Then when 200 threads is activated service do not reply to requests because is "max threads" in tomcat of service. > Why teiid do not send response as "Salesforce is down - error" or connection timeout? > Micro schema: > HTTP Requests to REST api -> Teiid spring boot app -> {broken 20min}Salesforce -- This message was sent by Atlassian Jira (v7.13.8#713008)

5 years, 5 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues June 2020