[JBoss JIRA] (TEIID-5965) Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5965?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5965:
---------------------------------------
Dropping to minor priority given the workaround and stopping to work on other issues.
> Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
> -------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-5965
> URL: https://issues.redhat.com/browse/TEIID-5965
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Minor
> Fix For: 15.0
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> In the specified example, the delimiter is TAB. Many web APIs allow customization of the delimiter character, and in order to provide a generic parser, sometimes it is way easier to define a delimiter as a variable, and not need to create a long nested structure with IF-THEN-ELSE-IF-ELSE constructs:
> {code:sql}
> Select * From TextTable (
> 'c1 c2
> 1 2'
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> {code}
> Imagine that based on setup on API side (out of our control) the content can be delivered via tab or semicolon, e.g.
> {code}
> c1;c2
> 1;2
> {code}
> or
> {code}
> c1 c2
> 1 2
> {code}
> Let's save this response into a variable and see the code, which we will need to write depending on the setup:
> {code:sql}
> Begin
> ...
> If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> End
> Else If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter ';'
> Header 1
> )x;
> End
> End
> {code}
> The if-else block is constantly growing, especially if we want to customize quote, escape, delimiter, and row delimiter.
> Thus it would be great if we could make these values configurable. In this case, we could end up with this expected code, which is more readable and more easily customizable:
> {code:sql}
> Begin
> ...
> Declare string delimiter = E'\t';
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter delimiter
> Header 1
> )x;
> End
> {code}
> With a bit of tweaking and certain assumptions, leading in the trust level, we can even read the first line and try to auto-detect the delimiter automatically (e.g. by counting tabs, commas, and semicolons in the first line).
> Please, could you be so kind as to make HEADER and SKIP values customizable (rather than hardcoded numbers)?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIID-5965) Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5965?page=com.atlassian.jira.plugi... ]
Work on TEIID-5965 stopped by Steven Hawkins.
---------------------------------------------
> Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
> -------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-5965
> URL: https://issues.redhat.com/browse/TEIID-5965
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> In the specified example, the delimiter is TAB. Many web APIs allow customization of the delimiter character, and in order to provide a generic parser, sometimes it is way easier to define a delimiter as a variable, and not need to create a long nested structure with IF-THEN-ELSE-IF-ELSE constructs:
> {code:sql}
> Select * From TextTable (
> 'c1 c2
> 1 2'
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> {code}
> Imagine that based on setup on API side (out of our control) the content can be delivered via tab or semicolon, e.g.
> {code}
> c1;c2
> 1;2
> {code}
> or
> {code}
> c1 c2
> 1 2
> {code}
> Let's save this response into a variable and see the code, which we will need to write depending on the setup:
> {code:sql}
> Begin
> ...
> If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> End
> Else If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter ';'
> Header 1
> )x;
> End
> End
> {code}
> The if-else block is constantly growing, especially if we want to customize quote, escape, delimiter, and row delimiter.
> Thus it would be great if we could make these values configurable. In this case, we could end up with this expected code, which is more readable and more easily customizable:
> {code:sql}
> Begin
> ...
> Declare string delimiter = E'\t';
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter delimiter
> Header 1
> )x;
> End
> {code}
> With a bit of tweaking and certain assumptions, leading in the trust level, we can even read the first line and try to auto-detect the delimiter automatically (e.g. by counting tabs, commas, and semicolons in the first line).
> Please, could you be so kind as to make HEADER and SKIP values customizable (rather than hardcoded numbers)?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIID-5965) Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5965?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5965:
----------------------------------
Priority: Minor (was: Major)
> Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
> -------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-5965
> URL: https://issues.redhat.com/browse/TEIID-5965
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Minor
> Fix For: 15.0
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> In the specified example, the delimiter is TAB. Many web APIs allow customization of the delimiter character, and in order to provide a generic parser, sometimes it is way easier to define a delimiter as a variable, and not need to create a long nested structure with IF-THEN-ELSE-IF-ELSE constructs:
> {code:sql}
> Select * From TextTable (
> 'c1 c2
> 1 2'
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> {code}
> Imagine that based on setup on API side (out of our control) the content can be delivered via tab or semicolon, e.g.
> {code}
> c1;c2
> 1;2
> {code}
> or
> {code}
> c1 c2
> 1 2
> {code}
> Let's save this response into a variable and see the code, which we will need to write depending on the setup:
> {code:sql}
> Begin
> ...
> If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> End
> Else If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter ';'
> Header 1
> )x;
> End
> End
> {code}
> The if-else block is constantly growing, especially if we want to customize quote, escape, delimiter, and row delimiter.
> Thus it would be great if we could make these values configurable. In this case, we could end up with this expected code, which is more readable and more easily customizable:
> {code:sql}
> Begin
> ...
> Declare string delimiter = E'\t';
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter delimiter
> Header 1
> )x;
> End
> {code}
> With a bit of tweaking and certain assumptions, leading in the trust level, we can even read the first line and try to auto-detect the delimiter automatically (e.g. by counting tabs, commas, and semicolons in the first line).
> Please, could you be so kind as to make HEADER and SKIP values customizable (rather than hardcoded numbers)?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIIDSB-212) ftp connection factory references ftp translator instead of file
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-212?page=com.atlassian.jira.plug... ]
Steven Hawkins resolved TEIIDSB-212.
------------------------------------
Resolution: Done
Corrected the translator names and dependencies and added a test to check that the translator specified actually exists.
We may want to add other checks including ensuring mustache files exist and that data projects get added as dependencies to the util project.
> ftp connection factory references ftp translator instead of file
> ----------------------------------------------------------------
>
> Key: TEIIDSB-212
> URL: https://issues.redhat.com/browse/TEIIDSB-212
> Project: Teiid Spring Boot
> Issue Type: Bug
> Affects Versions: 1.5.0
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.6.0
>
>
> A regression with TEIIDSB-186 was setting the translator name for ftp as ftp.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIIDSB-212) ftp connection factory references ftp translator instead of file
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-212?focusedWorklogId=12451581&pa... ]
Steven Hawkins logged work on TEIIDSB-212:
------------------------------------------
Author: Steven Hawkins
Created on: 25/Jun/20 9:13 AM
Start Date: 25/Jun/20 9:13 AM
Worklog Time Spent: 2 hours
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 2 hours)
Time Spent: 2 hours
Worklog Id: (was: 12451581)
> ftp connection factory references ftp translator instead of file
> ----------------------------------------------------------------
>
> Key: TEIIDSB-212
> URL: https://issues.redhat.com/browse/TEIIDSB-212
> Project: Teiid Spring Boot
> Issue Type: Bug
> Affects Versions: 1.5.0
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 2 hours
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> A regression with TEIIDSB-186 was setting the translator name for ftp as ftp.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIID-5965) Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5965?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5965:
---------------------------------------
The first problem I see with this is that the current grammar allows for HEADER to be specified without a value, which then defaults to 1. However this introduces an ambiguity into the grammar when we accept expressions the non-reserved keywork skip could mean the skip option or it could be the start of an expression.
Another thought is that even with the existing support if you are in the context of a procedure you can you dynamic sql to construct your parsing rather than an if/else chain.
> Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
> -------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-5965
> URL: https://issues.redhat.com/browse/TEIID-5965
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> In the specified example, the delimiter is TAB. Many web APIs allow customization of the delimiter character, and in order to provide a generic parser, sometimes it is way easier to define a delimiter as a variable, and not need to create a long nested structure with IF-THEN-ELSE-IF-ELSE constructs:
> {code:sql}
> Select * From TextTable (
> 'c1 c2
> 1 2'
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> {code}
> Imagine that based on setup on API side (out of our control) the content can be delivered via tab or semicolon, e.g.
> {code}
> c1;c2
> 1;2
> {code}
> or
> {code}
> c1 c2
> 1 2
> {code}
> Let's save this response into a variable and see the code, which we will need to write depending on the setup:
> {code:sql}
> Begin
> ...
> If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> End
> Else If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter ';'
> Header 1
> )x;
> End
> End
> {code}
> The if-else block is constantly growing, especially if we want to customize quote, escape, delimiter, and row delimiter.
> Thus it would be great if we could make these values configurable. In this case, we could end up with this expected code, which is more readable and more easily customizable:
> {code:sql}
> Begin
> ...
> Declare string delimiter = E'\t';
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter delimiter
> Header 1
> )x;
> End
> {code}
> With a bit of tweaking and certain assumptions, leading in the trust level, we can even read the first line and try to auto-detect the delimiter automatically (e.g. by counting tabs, commas, and semicolons in the first line).
> Please, could you be so kind as to make HEADER and SKIP values customizable (rather than hardcoded numbers)?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIID-5950) Create an athena translator
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5950?page=com.atlassian.jira.plugi... ]
Work on TEIID-5950 started by Steven Hawkins.
---------------------------------------------
> Create an athena translator
> ---------------------------
>
> Key: TEIID-5950
> URL: https://issues.redhat.com/browse/TEIID-5950
> Project: Teiid
> Issue Type: Enhancement
> Components: JDBC Connector
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 4 hours
> Remaining Estimate: 4 hours
>
> There have been several issues seen with athena using the jdbc ansi translator - subquery support and bind variables specifically. There should be a specific translator so that we can account for those better.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months
[JBoss JIRA] (TEIID-5965) Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5965?page=com.atlassian.jira.plugi... ]
Work on TEIID-5965 started by Steven Hawkins.
---------------------------------------------
> Allow variables to be used as TextTable delimeters, row delimeters, quote, header, skip rows, and escape characters
> -------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-5965
> URL: https://issues.redhat.com/browse/TEIID-5965
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> In the specified example, the delimiter is TAB. Many web APIs allow customization of the delimiter character, and in order to provide a generic parser, sometimes it is way easier to define a delimiter as a variable, and not need to create a long nested structure with IF-THEN-ELSE-IF-ELSE constructs:
> {code:sql}
> Select * From TextTable (
> 'c1 c2
> 1 2'
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> {code}
> Imagine that based on setup on API side (out of our control) the content can be delivered via tab or semicolon, e.g.
> {code}
> c1;c2
> 1;2
> {code}
> or
> {code}
> c1 c2
> 1 2
> {code}
> Let's save this response into a variable and see the code, which we will need to write depending on the setup:
> {code:sql}
> Begin
> ...
> If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter E'\t'
> Header 1
> )x;
> End
> Else If (delimiter = 'tab')
> Begin
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter ';'
> Header 1
> )x;
> End
> End
> {code}
> The if-else block is constantly growing, especially if we want to customize quote, escape, delimiter, and row delimiter.
> Thus it would be great if we could make these values configurable. In this case, we could end up with this expected code, which is more readable and more easily customizable:
> {code:sql}
> Begin
> ...
> Declare string delimiter = E'\t';
> Select * From TextTable (
> apiResponse
> Columns
> c1 integer,
> c2 integer
> Delimiter delimiter
> Header 1
> )x;
> End
> {code}
> With a bit of tweaking and certain assumptions, leading in the trust level, we can even read the first line and try to auto-detect the delimiter automatically (e.g. by counting tabs, commas, and semicolons in the first line).
> Please, could you be so kind as to make HEADER and SKIP values customizable (rather than hardcoded numbers)?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 11 months