[JBoss JIRA] (TEIID-2896) Add TEXTTABLE function capabilities to interpret fixed width data file content that is shorter than the defined line length
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-2896?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-2896:
------------------------------------
2GALT.RO09 ALTERNATE CC DATAAAAAA (2009) null 0 0000000000000000 ALT.RO09
I've replaced the empty location with null about, as to where it aligns with the rows below it.
> Add TEXTTABLE function capabilities to interpret fixed width data file content that is shorter than the defined line length
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-2896
> URL: https://issues.jboss.org/browse/TEIID-2896
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 8.7
>
> Attachments: sample, TEIID-2896.patch
>
>
> Current behavior is that data lines too short to fit a fixed width data interpretation produces an error such as "Error: Text parse error: Fixed width line width 35 is smaller than the expected 60 on text line 1 in file:/tmp/file.csv.". The customer's use case is interpreting data files from another application which bundles multiple types of data along with additional descriptive info/metadata about the data contained within the file. As the error is thrown if any lines after the first interpreted line are shorter than the summed value of the fixed width columns, they are unable to query the file. The customer wishes for the ERROR to be suppressed and the query to return the data independent of whether it fits the fixed width criteria and potentially to be able to filter the data being retrieved.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2988) MongoDB: Function substring - starting index and number of arguments
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2988?page=com.atlassian.jira.plugin... ]
Ramesh Reddy updated TEIID-2988:
--------------------------------
Description:
Two minor bugs in function substring()
Mongo's function $substr takes only 3 arguments so this query fails:
{code}
SELECT intkey, SUBSTRING(BQT1.SmallA.stringkey, 1) FROM BQT1.SmallA ORDER BY intkey
{code}
{code}
Exception:13:18:48,220 ERROR [org.teiid.CONNECTOR] (Worker38_QueryProcessorQueue345) Connector worker process failed for atomic-request=omsmHP0o16Sa.40.0.86: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: the $substr operator requires 3 operand(s)" , "code" : 16020 , "ok" : 0.0}
at com.mongodb.CommandResult.getException(CommandResult.java:71)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:110)
at com.mongodb.DBCollection.aggregate(DBCollection.java:1308)
{code}
Generated mongo command:
{code}
{ aggregate: "smalla", pipeline: [ { $project: { c_0: "$INTKEY", c_1: { $substr: [ "$STRINGKEY", 1 ] } } }, { $sort: { c_0: 1 } } ] }
{code}
Mongo's $substr function second argument represents number of bytes to skip, not the starting index. The function substring(StringKey,1,1) returns the second letter, but it should return the first letter.
was:
Two minor bugs in function substring()
Mongo's function $substr takes only 3 arguments so this query fails:
SELECT intkey, SUBSTRING(BQT1.SmallA.stringkey, 1) FROM BQT1.SmallA ORDER BY intkey
Exception:13:18:48,220 ERROR [org.teiid.CONNECTOR] (Worker38_QueryProcessorQueue345) Connector worker process failed for atomic-request=omsmHP0o16Sa.40.0.86: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: the $substr operator requires 3 operand(s)" , "code" : 16020 , "ok" : 0.0}
at com.mongodb.CommandResult.getException(CommandResult.java:71)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:110)
at com.mongodb.DBCollection.aggregate(DBCollection.java:1308)
Generated mongo command:
{ aggregate: "smalla", pipeline: [ { $project: { c_0: "$INTKEY", c_1: { $substr: [ "$STRINGKEY", 1 ] } } }, { $sort: { c_0: 1 } } ] }
Mongo's $substr function second argument represents number of bytes to skip, not the starting index. The function substring(StringKey,1,1) returns the second letter, but it should return the first letter.
> MongoDB: Function substring - starting index and number of arguments
> --------------------------------------------------------------------
>
> Key: TEIID-2988
> URL: https://issues.jboss.org/browse/TEIID-2988
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.4.1
> Reporter: Filip Elias
> Assignee: Ramesh Reddy
> Fix For: 8.7.1, 8.8
>
>
> Two minor bugs in function substring()
> Mongo's function $substr takes only 3 arguments so this query fails:
> {code}
> SELECT intkey, SUBSTRING(BQT1.SmallA.stringkey, 1) FROM BQT1.SmallA ORDER BY intkey
> {code}
> {code}
> Exception:13:18:48,220 ERROR [org.teiid.CONNECTOR] (Worker38_QueryProcessorQueue345) Connector worker process failed for atomic-request=omsmHP0o16Sa.40.0.86: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: the $substr operator requires 3 operand(s)" , "code" : 16020 , "ok" : 0.0}
> at com.mongodb.CommandResult.getException(CommandResult.java:71)
> at com.mongodb.CommandResult.throwOnError(CommandResult.java:110)
> at com.mongodb.DBCollection.aggregate(DBCollection.java:1308)
> {code}
> Generated mongo command:
> {code}
> { aggregate: "smalla", pipeline: [ { $project: { c_0: "$INTKEY", c_1: { $substr: [ "$STRINGKEY", 1 ] } } }, { $sort: { c_0: 1 } } ] }
> {code}
> Mongo's $substr function second argument represents number of bytes to skip, not the starting index. The function substring(StringKey,1,1) returns the second letter, but it should return the first letter.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2896) Add TEXTTABLE function capabilities to interpret fixed width data file content that is shorter than the defined line length
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2896?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2896:
---------------------------------------
> You don't by chance have the TEXTTABLE that you specified to get the sample file to work?
You can also look at the unit test added with the commit.
> After reading the documentation, it says fixed width files are not supported when using SELECTOR
Which documentation are you reading? With the 8.7 and later docs it should indicate that you can do this.
> Add TEXTTABLE function capabilities to interpret fixed width data file content that is shorter than the defined line length
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-2896
> URL: https://issues.jboss.org/browse/TEIID-2896
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 8.7
>
> Attachments: sample, TEIID-2896.patch
>
>
> Current behavior is that data lines too short to fit a fixed width data interpretation produces an error such as "Error: Text parse error: Fixed width line width 35 is smaller than the expected 60 on text line 1 in file:/tmp/file.csv.". The customer's use case is interpreting data files from another application which bundles multiple types of data along with additional descriptive info/metadata about the data contained within the file. As the error is thrown if any lines after the first interpreted line are shorter than the summed value of the fixed width columns, they are unable to query the file. The customer wishes for the ERROR to be suppressed and the query to return the data independent of whether it fits the fixed width criteria and potentially to be able to filter the data being retrieved.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2896) Add TEXTTABLE function capabilities to interpret fixed width data file content that is shorter than the defined line length
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-2896?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-2896:
------------------------------------
You don't by chance have the TEXTTABLE that you specified to get the sample file to work?
After reading the documentation, it says fixed width files are not supported when using SELECTOR. So if you have an example of how you accomplished this, it would be appreciated.
> Add TEXTTABLE function capabilities to interpret fixed width data file content that is shorter than the defined line length
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-2896
> URL: https://issues.jboss.org/browse/TEIID-2896
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 8.7
>
> Attachments: sample, TEIID-2896.patch
>
>
> Current behavior is that data lines too short to fit a fixed width data interpretation produces an error such as "Error: Text parse error: Fixed width line width 35 is smaller than the expected 60 on text line 1 in file:/tmp/file.csv.". The customer's use case is interpreting data files from another application which bundles multiple types of data along with additional descriptive info/metadata about the data contained within the file. As the error is thrown if any lines after the first interpreted line are shorter than the summed value of the fixed width columns, they are unable to query the file. The customer wishes for the ERROR to be suppressed and the query to return the data independent of whether it fits the fixed width criteria and potentially to be able to filter the data being retrieved.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2983) MongoDB: Function "count()" returns sum instead of number of rows
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2983?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2983:
---------------------------------------
> Is there a way re-write multiple aggregate statement above? right now I can only think that that needs to split into two statements.
Yes, unless you can use subqueries the only other alternative would be a more complicated sum expression:
sum(case when col is not null then 1 else 0 end)
> MongoDB can group by multiple elements in a group, then we can do a sum like
The grouping should come from the group by correct?
> MongoDB: Function "count()" returns sum instead of number of rows
> -----------------------------------------------------------------
>
> Key: TEIID-2983
> URL: https://issues.jboss.org/browse/TEIID-2983
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Van Halbert
> Assignee: Ramesh Reddy
> Labels: Beta1
> Fix For: 8.7.1, 8.8
>
>
> Function "count()" returns sum of rows instead of number of rows
> Query [1] returns sum of values of all rows of IntKey column.
> Query [2] correctly returns number of rows
>
> [1] SELECT COUNT(IntKey) FROM BQT1.SmallA
> [2] SELECT COUNT(*) FROM BQT1.SmallA
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2986) issues with embedded
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2986?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2986:
---------------------------------------
Should this be applied to 8.7.x as well?
> issues with embedded
> --------------------
>
> Key: TEIID-2986
> URL: https://issues.jboss.org/browse/TEIID-2986
> Project: Teiid
> Issue Type: Bug
> Components: Embedded
> Affects Versions: 8.7
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.7.1, 8.8
>
>
> The dependency set for the ws translator is incomplete the cxf bundle jar and several other dependencies are needed.
> We are not including the JBoss modules jar, but have an implied dependency through EmbeddedProfile, which should be refactored so that the modules jar is not needed in the embedded kit.
> Also the classpaths in the example scripts are incorrect with the 8.7 refactoring to put optional dependencies in subfolders.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2990) MongoDB: Usage of lookup() function throws an exception
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2990?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2990:
---------------------------------------
lookup is not being pushed and this isn't really related to lookup. Rather it's the source query "SELECT 1 FROM BQT1.smallb" that is causing mongo to indicate that "... $projection requires at least one output field"
> MongoDB: Usage of lookup() function throws an exception
> -------------------------------------------------------
>
> Key: TEIID-2990
> URL: https://issues.jboss.org/browse/TEIID-2990
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.4.1
> Reporter: Filip Elias
> Assignee: Ramesh Reddy
> Fix For: 8.7.1, 8.8
>
> Attachments: server.log
>
>
> Exception[1] is thrown when function lookup() is used in SQL query.
> Mongo translator doesn't support selection of constants only, which causes the exception(select 1, select 'hi' from smalla).
> [1] 3:59:20,997 ERROR [org.teiid.CONNECTOR] (Worker5_QueryProcessorQueue6) Connector worker process failed for atomic-request=Ef2Z7E0246by.1.5.3: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: $projection requires at least one output field" , "code" : 16403 , "ok" : 0.0}
> at com.mongodb.CommandResult.getException(CommandResult.java:71)
> Server log is in the attachment.
> Sample query:
> SELECT BQT1.SmallA.IntKey FROM BQT1.SmallA, BQT1.SmallB WHERE BQT1.SmallA.IntKey= lookup('BQT1.SmallB', 'IntKey', 'StringKey', BQT1.SmallA.StringKey) ORDER BY IntKey
> Generated mongo commands:
> { aggregate: "smalla", pipeline: [ { $project: { _m0: "$INTKEY", _m1: "$STRINGKEY" } } ] }
> and
> { aggregate: "smallb", pipeline: [ { $project: {} } ] }
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2990) MongoDB: Usage of lookup() function throws an exception
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2990?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2990:
-------------------------------------
lookup is not defined as supported functions, I am not sure how this being pushed to the translator
> MongoDB: Usage of lookup() function throws an exception
> -------------------------------------------------------
>
> Key: TEIID-2990
> URL: https://issues.jboss.org/browse/TEIID-2990
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.4.1
> Reporter: Filip Elias
> Assignee: Ramesh Reddy
> Fix For: 8.7.1, 8.8
>
> Attachments: server.log
>
>
> Exception[1] is thrown when function lookup() is used in SQL query.
> Mongo translator doesn't support selection of constants only, which causes the exception(select 1, select 'hi' from smalla).
> [1] 3:59:20,997 ERROR [org.teiid.CONNECTOR] (Worker5_QueryProcessorQueue6) Connector worker process failed for atomic-request=Ef2Z7E0246by.1.5.3: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: $projection requires at least one output field" , "code" : 16403 , "ok" : 0.0}
> at com.mongodb.CommandResult.getException(CommandResult.java:71)
> Server log is in the attachment.
> Sample query:
> SELECT BQT1.SmallA.IntKey FROM BQT1.SmallA, BQT1.SmallB WHERE BQT1.SmallA.IntKey= lookup('BQT1.SmallB', 'IntKey', 'StringKey', BQT1.SmallA.StringKey) ORDER BY IntKey
> Generated mongo commands:
> { aggregate: "smalla", pipeline: [ { $project: { _m0: "$INTKEY", _m1: "$STRINGKEY" } } ] }
> and
> { aggregate: "smallb", pipeline: [ { $project: {} } ] }
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2990) MongoDB: Usage of lookup() function throws an exception
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-2990?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration updated TEIID-2990:
-------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1105121
> MongoDB: Usage of lookup() function throws an exception
> -------------------------------------------------------
>
> Key: TEIID-2990
> URL: https://issues.jboss.org/browse/TEIID-2990
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.4.1
> Reporter: Filip Elias
> Assignee: Ramesh Reddy
> Fix For: 8.7.1, 8.8
>
> Attachments: server.log
>
>
> Exception[1] is thrown when function lookup() is used in SQL query.
> Mongo translator doesn't support selection of constants only, which causes the exception(select 1, select 'hi' from smalla).
> [1] 3:59:20,997 ERROR [org.teiid.CONNECTOR] (Worker5_QueryProcessorQueue6) Connector worker process failed for atomic-request=Ef2Z7E0246by.1.5.3: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: $projection requires at least one output field" , "code" : 16403 , "ok" : 0.0}
> at com.mongodb.CommandResult.getException(CommandResult.java:71)
> Server log is in the attachment.
> Sample query:
> SELECT BQT1.SmallA.IntKey FROM BQT1.SmallA, BQT1.SmallB WHERE BQT1.SmallA.IntKey= lookup('BQT1.SmallB', 'IntKey', 'StringKey', BQT1.SmallA.StringKey) ORDER BY IntKey
> Generated mongo commands:
> { aggregate: "smalla", pipeline: [ { $project: { _m0: "$INTKEY", _m1: "$STRINGKEY" } } ] }
> and
> { aggregate: "smallb", pipeline: [ { $project: {} } ] }
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years
[JBoss JIRA] (TEIID-2990) MongoDB: Usage of lookup() function throws an exception
by Filip Elias (JIRA)
[ https://issues.jboss.org/browse/TEIID-2990?page=com.atlassian.jira.plugin... ]
Filip Elias updated TEIID-2990:
-------------------------------
Attachment: server.log
> MongoDB: Usage of lookup() function throws an exception
> -------------------------------------------------------
>
> Key: TEIID-2990
> URL: https://issues.jboss.org/browse/TEIID-2990
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.4.1
> Reporter: Filip Elias
> Assignee: Ramesh Reddy
> Fix For: 8.7.1, 8.8
>
> Attachments: server.log
>
>
> Exception[1] is thrown when function lookup() is used in SQL query.
> Mongo translator doesn't support selection of constants only, which causes the exception(select 1, select 'hi' from smalla).
> [1] 3:59:20,997 ERROR [org.teiid.CONNECTOR] (Worker5_QueryProcessorQueue6) Connector worker process failed for atomic-request=Ef2Z7E0246by.1.5.3: com.mongodb.CommandFailureException: { "serverUsed" : "localhost/127.0.0.1:27017" , "errmsg" : "exception: $projection requires at least one output field" , "code" : 16403 , "ok" : 0.0}
> at com.mongodb.CommandResult.getException(CommandResult.java:71)
> Server log is in the attachment.
> Sample query:
> SELECT BQT1.SmallA.IntKey FROM BQT1.SmallA, BQT1.SmallB WHERE BQT1.SmallA.IntKey= lookup('BQT1.SmallB', 'IntKey', 'StringKey', BQT1.SmallA.StringKey) ORDER BY IntKey
> Generated mongo commands:
> { aggregate: "smalla", pipeline: [ { $project: { _m0: "$INTKEY", _m1: "$STRINGKEY" } } ] }
> and
> { aggregate: "smallb", pipeline: [ { $project: {} } ] }
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years