[JBoss JIRA] (TEIID-3647) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-3647?focusedWorklogId=12451553&pag... ]
Steven Hawkins logged work on TEIID-3647:
-----------------------------------------
Author: Steven Hawkins
Created on: 22/Jun/20 10:39 AM
Start Date: 22/Jun/20 10:39 AM
Worklog Time Spent: 6 hours
Work Description: Time spent switching dependencies, updating accumulo, and fixing classloading issues.
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 1 hour, 30 minutes)
Time Spent: 1 day, 2 hours, 30 minutes (was: 4 hours, 30 minutes)
Worklog Id: (was: 12451553)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIID-3647
> URL: https://issues.redhat.com/browse/TEIID-3647
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 6 hours
> Time Spent: 1 day, 2 hours, 30 minutes
> Remaining Estimate: 0 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5949) SQLServer DateTimeOffset datatype is read as object(34)
by Manoj Majumdar (Jira)
[ https://issues.redhat.com/browse/TEIID-5949?page=com.atlassian.jira.plugi... ]
Manoj Majumdar edited comment on TEIID-5949 at 6/22/20 5:15 AM:
----------------------------------------------------------------
I altered the source metadata to read it as string, and used preparedStatement.setObject, it is working in every scenario for this combination. Reading it as timestamp was causing loss of precision and offset , as well I was unable to use it for comparision in "SELECT" query.
was (Author: manoj1492):
I altered the source metadata to read it as string, and used preparedStatement.setObject, it is working in every scenario for this combination
> SQLServer DateTimeOffset datatype is read as object(34)
> -------------------------------------------------------
>
> Key: TEIID-5949
> URL: https://issues.redhat.com/browse/TEIID-5949
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Driver
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Labels: TEIID30504, sqlserver2012
> Fix For: 15.0
>
>
> Hi,
> I am trying to set null in datetimeoffset datatype of SQLSERVER using Teiid , but it is giving the error as:
> org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 B_MSSQL1: S0003 Implicit conversion from data type varbinary to datetimeoffset is not allowed. Use the CONVERT function to run this query.
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:752)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:562)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:571)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:296)
> I am just using a table, with only one column "datetimeoffset" and when I an using Preparedstatement.setNull(1, Types.TIMESTAMP), it is throwing this error.
> Also the datetimeoffset datatype is read as "object(34)" while fetching the schema using Teiid and Microsoft JDBC driver.
> Please suggest any ways to overcome this issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5949) SQLServer DateTimeOffset datatype is read as object(34)
by Manoj Majumdar (Jira)
[ https://issues.redhat.com/browse/TEIID-5949?page=com.atlassian.jira.plugi... ]
Manoj Majumdar commented on TEIID-5949:
---------------------------------------
I altered the source metadata to read it as string, and used preparedStatement.setObject, it is working in every scenario for this combination
> SQLServer DateTimeOffset datatype is read as object(34)
> -------------------------------------------------------
>
> Key: TEIID-5949
> URL: https://issues.redhat.com/browse/TEIID-5949
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Driver
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Labels: TEIID30504, sqlserver2012
> Fix For: 15.0
>
>
> Hi,
> I am trying to set null in datetimeoffset datatype of SQLSERVER using Teiid , but it is giving the error as:
> org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 B_MSSQL1: S0003 Implicit conversion from data type varbinary to datetimeoffset is not allowed. Use the CONVERT function to run this query.
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:752)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:562)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:571)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:296)
> I am just using a table, with only one column "datetimeoffset" and when I an using Preparedstatement.setNull(1, Types.TIMESTAMP), it is throwing this error.
> Also the datetimeoffset datatype is read as "object(34)" while fetching the schema using Teiid and Microsoft JDBC driver.
> Please suggest any ways to overcome this issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5975) xref links are broken in docs and other doc issues
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5975?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5975:
---------------------------------------
> They produce an invalid anchor link in the current doc.
I don't see a great workaround here. Updating the links to be external will break the product build and it will be a lot of work to instead change the current gitbook to perform all of the includes that effectively inline content that the product build does. That unfortunately leaves us with using templating to produce a different link depending on the build target.
> xref links are broken in docs and other doc issues
> --------------------------------------------------
>
> Key: TEIID-5975
> URL: https://issues.redhat.com/browse/TEIID-5975
> Project: Teiid
> Issue Type: Bug
> Components: Documentation
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
>
> Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
> There are also some pages that look like they weren't updated to the new page names. Running a regex like
> {noformat}
> :[^/A-Z:]*[A-Z][^/:]*.adoc
> {noformat}
> over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5975) xref links are broken in docs and other doc issues
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5975?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5975:
----------------------------------
Description:
Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
There are also some pages that look like they weren't updated to the new page names. Running a regex like
{noformat}
:[^/A-Z:]*[A-Z][^/:]*.adoc
{noformat}
over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
was:
Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
There are also some pages that look like they weren't updated to the new page names. Running a regex like :[^/A-Z:]*[A-Z][^/:]*.adoc over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
> xref links are broken in docs and other doc issues
> --------------------------------------------------
>
> Key: TEIID-5975
> URL: https://issues.redhat.com/browse/TEIID-5975
> Project: Teiid
> Issue Type: Bug
> Components: Documentation
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
>
> Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
> There are also some pages that look like they weren't updated to the new page names. Running a regex like
> {noformat}
> :[^/A-Z:]*[A-Z][^/:]*.adoc
> {noformat}
> over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5975) xref links are broken in docs and other doc issues
by Steven Hawkins (Jira)
Steven Hawkins created TEIID-5975:
-------------------------------------
Summary: xref links are broken in docs and other doc issues
Key: TEIID-5975
URL: https://issues.redhat.com/browse/TEIID-5975
Project: Teiid
Issue Type: Bug
Components: Documentation
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
There are also some pages that look like they weren't updated to the new page names. Running a regex like :[^/A-Z:]*[A-Z][^/:]*.adoc over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-3647) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-3647?focusedWorklogId=12451527&pag... ]
Steven Hawkins logged work on TEIID-3647:
-----------------------------------------
Author: Steven Hawkins
Created on: 18/Jun/20 8:51 AM
Start Date: 18/Jun/20 8:49 AM
Worklog Time Spent: 4 hours, 30 minutes
Work Description: Time spent porting the implementation to wildfly and aligning the dependencies.
Issue Time Tracking
-------------------
Remaining Estimate: 1 hour, 30 minutes (was: 6 hours)
Time Spent: 4 hours, 30 minutes
Worklog Id: (was: 12451527)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIID-3647
> URL: https://issues.redhat.com/browse/TEIID-3647
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 6 hours
> Time Spent: 4 hours, 30 minutes
> Remaining Estimate: 1 hour, 30 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIIDSB-206) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-206?focusedWorklogId=12451526&pa... ]
Steven Hawkins logged work on TEIIDSB-206:
------------------------------------------
Author: Steven Hawkins
Created on: 18/Jun/20 8:49 AM
Start Date: 18/Jun/20 8:48 AM
Worklog Time Spent: 1 hour
Work Description: Additional time walking through Aditya's work.
Issue Time Tracking
-------------------
Remaining Estimate: 2 days, 30 minutes (was: 2 days, 1 hour, 30 minutes)
Time Spent: 3 hours, 30 minutes (was: 2 hours, 30 minutes)
Worklog Id: (was: 12451526)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIIDSB-206
> URL: https://issues.redhat.com/browse/TEIIDSB-206
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Components: datasource
> Reporter: Van Halbert
> Assignee: Aditya Manglam Sharma
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 2 days, 4 hours
> Time Spent: 3 hours, 30 minutes
> Remaining Estimate: 2 days, 30 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-3647) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-3647?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-3647:
----------------------------------
Original Estimate: 6 hours
Remaining Estimate: 6 hours
Story Points: 1
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIID-3647
> URL: https://issues.redhat.com/browse/TEIID-3647
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 6 hours
> Remaining Estimate: 6 hours
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5969) The "already removed" exception in case of copyLobs=true translator property for MSSQL
by Dmitrii Pogorelov (Jira)
[ https://issues.redhat.com/browse/TEIID-5969?page=com.atlassian.jira.plugi... ]
Dmitrii Pogorelov commented on TEIID-5969:
------------------------------------------
[~shawkins] thx a lot for the quick fix, I've already merged your changes. It helped to solve the "already removed" problem.
> The "already removed" exception in case of copyLobs=true translator property for MSSQL
> --------------------------------------------------------------------------------------
>
> Key: TEIID-5969
> URL: https://issues.redhat.com/browse/TEIID-5969
> Project: Teiid
> Issue Type: Quality Risk
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0, 13.1.2, 14.0.1
>
> Original Estimate: 2 hours
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> We're getting the following exception:
> {code:java}
> 2020-06-10 16:01:45,326 WARN [org.teiid.PROCESSOR] (Worker21_QueryProcessorQueue272) 1bxj3NJqJLpO TEIID30020 Processing exception for request 1bxj3NJqJLpO.10 'TEIID30328 Unable to evaluate convert(ms_dwh.v1.expr1, string): TEIID30384 Error while evaluating function convert'. Originally ExpressionEvaluationException 'already removed' FileStore.java:156. Enable more detailed logging to see the entire stacktrace. {code}
> in the case of MSSQL connector and copyLobs=true in its translator property.
> After debugging I discovered that the error occurred actually because of not enough value in
> MAX_LOB_MEMORY_BYTES which is used in LobManager. The variable depends on
> org.teiid.maxStringLength which is set usually to 4000 by default. But if someone put a data with text field type in MSSQL having more than, say, 20000 symbols then the field won't be read by Teiid at all, Teiid will just throw out the exception mentioned above. Will it be possible to improve LobManager so that it can read Text data dynamically increasing needed memory for the process? Otherwise even setting the org.teiid.maxStringLength to 20000 Teiid can raise the exception because of a Text data with length more than 20000 symbols.
>
> Steps to reproduce:
> # create the v1 table in MSSQL:
> {code:java}
> CREATE TABLE [dbo].[v1](CREATE TABLE [dbo].[v1]( [expr1] [text] NULL, [b] [int] NULL) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] {code}
> 2. insert in the table the following data (I did it via standard MSSQL Server Management Studio):
> {code:java}
> insert into test_dwh.dbo.v1 values (replicate(convert(varchar(max),'a'), 20000), 1) ; {code}
> 3. add MSSQL as a source to Teiid:
> {code:java}
> <datasource jndi-name="java:/mssql-test-dwh" pool-name="mssql-test-dwh" enabled="true" use-java-context="true">
> <connection-url>jdbc:sqlserver://localhost:1433;databaseName=test_dwh</connection-url>
> <driver-class&gt;com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class&gt;
> <driver>com.microsoft.sqlserver</driver>
> <new-connection-sql>SET ANSI_WARNINGS OFF</new-connection-sql>
> <pool>
> <min-pool-size>2</min-pool-size>
> <max-pool-size>70</max-pool-size>
> </pool>
> <security>
> <user-name>*****</user-name>
> <password>*****</password>
> </security>
> <validation>
> <check-valid-connection-sql>select 1</check-valid-connection-sql>
> </validation>
> <timeout>
> <blocking-timeout-millis>120000</blocking-timeout-millis>
> <idle-timeout-minutes>5</idle-timeout-minutes>
> </timeout>
> </datasource>{code}
> 4. add ms model:
> {code:java}
> <model name="ms_dwh">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="importer.tableTypes" value="TABLE,VIEW"/>
> <property name="importer.importKeys" value="false"/>
> <property name="importer.schemaPattern" value="dbo"/>
> <source name="dwh_ms" translator-name="mySqlserver" connection-jndi-name="java:/mssql-test-dwh"/>
> </model>{code}
> 5. add mySqlserver translator:
> {code:java}
> <translator name="mySqlserver" type="sqlserver">
> <property name="SupportsNativeQueries" value="true"/>
> <property name="copyLobs" value="true"/>
> </translator> {code}
> 6. try to run the following query:
> {code:java}
> select cast(expr1 as string) from ms_dwh.v1 ;; {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years