[JBoss JIRA] (TEIIDSB-206) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-206?page=com.atlassian.jira.plug... ]
Steven Hawkins commented on TEIIDSB-206:
----------------------------------------
The pr for aligning these changes with the updates from Teiid has been opened.
[~rareddy] how do you want to handle operator support for this? Namely we have an excel and file sources that presume a file connection. Do you want to also add a named excel-hdfs and file-hdfs (likewise we are currently missing the ftp variants) or should we look to make things a bit more flexible / abstract and look toward using a convention, ddl, or datasource properties to know the source type? This will continue to fan out with more file sources, such as s3, and more file consumers, such as avro (that could mean creating avro-ftp, avro-hdfs, avro-file, avro-s3)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIIDSB-206
> URL: https://issues.redhat.com/browse/TEIIDSB-206
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Components: datasource
> Reporter: Van Halbert
> Assignee: Aditya Manglam Sharma
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 2 days, 4 hours
> Time Spent: 3 hours, 30 minutes
> Remaining Estimate: 2 days, 30 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-3647) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-3647?focusedWorklogId=12451553&pag... ]
Steven Hawkins logged work on TEIID-3647:
-----------------------------------------
Author: Steven Hawkins
Created on: 22/Jun/20 10:39 AM
Start Date: 22/Jun/20 10:39 AM
Worklog Time Spent: 6 hours
Work Description: Time spent switching dependencies, updating accumulo, and fixing classloading issues.
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 1 hour, 30 minutes)
Time Spent: 1 day, 2 hours, 30 minutes (was: 4 hours, 30 minutes)
Worklog Id: (was: 12451553)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIID-3647
> URL: https://issues.redhat.com/browse/TEIID-3647
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 6 hours
> Time Spent: 1 day, 2 hours, 30 minutes
> Remaining Estimate: 0 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-5949) SQLServer DateTimeOffset datatype is read as object(34)
by Manoj Majumdar (Jira)
[ https://issues.redhat.com/browse/TEIID-5949?page=com.atlassian.jira.plugi... ]
Manoj Majumdar edited comment on TEIID-5949 at 6/22/20 5:15 AM:
----------------------------------------------------------------
I altered the source metadata to read it as string, and used preparedStatement.setObject, it is working in every scenario for this combination. Reading it as timestamp was causing loss of precision and offset , as well I was unable to use it for comparision in "SELECT" query.
was (Author: manoj1492):
I altered the source metadata to read it as string, and used preparedStatement.setObject, it is working in every scenario for this combination
> SQLServer DateTimeOffset datatype is read as object(34)
> -------------------------------------------------------
>
> Key: TEIID-5949
> URL: https://issues.redhat.com/browse/TEIID-5949
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Driver
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Labels: TEIID30504, sqlserver2012
> Fix For: 15.0
>
>
> Hi,
> I am trying to set null in datetimeoffset datatype of SQLSERVER using Teiid , but it is giving the error as:
> org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 B_MSSQL1: S0003 Implicit conversion from data type varbinary to datetimeoffset is not allowed. Use the CONVERT function to run this query.
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:752)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:562)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:571)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:296)
> I am just using a table, with only one column "datetimeoffset" and when I an using Preparedstatement.setNull(1, Types.TIMESTAMP), it is throwing this error.
> Also the datetimeoffset datatype is read as "object(34)" while fetching the schema using Teiid and Microsoft JDBC driver.
> Please suggest any ways to overcome this issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-5949) SQLServer DateTimeOffset datatype is read as object(34)
by Manoj Majumdar (Jira)
[ https://issues.redhat.com/browse/TEIID-5949?page=com.atlassian.jira.plugi... ]
Manoj Majumdar commented on TEIID-5949:
---------------------------------------
I altered the source metadata to read it as string, and used preparedStatement.setObject, it is working in every scenario for this combination
> SQLServer DateTimeOffset datatype is read as object(34)
> -------------------------------------------------------
>
> Key: TEIID-5949
> URL: https://issues.redhat.com/browse/TEIID-5949
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Driver
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Labels: TEIID30504, sqlserver2012
> Fix For: 15.0
>
>
> Hi,
> I am trying to set null in datetimeoffset datatype of SQLSERVER using Teiid , but it is giving the error as:
> org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 B_MSSQL1: S0003 Implicit conversion from data type varbinary to datetimeoffset is not allowed. Use the CONVERT function to run this query.
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:752)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:562)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:571)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:296)
> I am just using a table, with only one column "datetimeoffset" and when I an using Preparedstatement.setNull(1, Types.TIMESTAMP), it is throwing this error.
> Also the datetimeoffset datatype is read as "object(34)" while fetching the schema using Teiid and Microsoft JDBC driver.
> Please suggest any ways to overcome this issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-5975) xref links are broken in docs and other doc issues
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5975?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5975:
---------------------------------------
> They produce an invalid anchor link in the current doc.
I don't see a great workaround here. Updating the links to be external will break the product build and it will be a lot of work to instead change the current gitbook to perform all of the includes that effectively inline content that the product build does. That unfortunately leaves us with using templating to produce a different link depending on the build target.
> xref links are broken in docs and other doc issues
> --------------------------------------------------
>
> Key: TEIID-5975
> URL: https://issues.redhat.com/browse/TEIID-5975
> Project: Teiid
> Issue Type: Bug
> Components: Documentation
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
>
> Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
> There are also some pages that look like they weren't updated to the new page names. Running a regex like
> {noformat}
> :[^/A-Z:]*[A-Z][^/:]*.adoc
> {noformat}
> over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-5975) xref links are broken in docs and other doc issues
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5975?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5975:
----------------------------------
Description:
Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
There are also some pages that look like they weren't updated to the new page names. Running a regex like
{noformat}
:[^/A-Z:]*[A-Z][^/:]*.adoc
{noformat}
over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
was:
Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
There are also some pages that look like they weren't updated to the new page names. Running a regex like :[^/A-Z:]*[A-Z][^/:]*.adoc over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
> xref links are broken in docs and other doc issues
> --------------------------------------------------
>
> Key: TEIID-5975
> URL: https://issues.redhat.com/browse/TEIID-5975
> Project: Teiid
> Issue Type: Bug
> Components: Documentation
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
>
> Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
> There are also some pages that look like they weren't updated to the new page names. Running a regex like
> {noformat}
> :[^/A-Z:]*[A-Z][^/:]*.adoc
> {noformat}
> over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-5975) xref links are broken in docs and other doc issues
by Steven Hawkins (Jira)
Steven Hawkins created TEIID-5975:
-------------------------------------
Summary: xref links are broken in docs and other doc issues
Key: TEIID-5975
URL: https://issues.redhat.com/browse/TEIID-5975
Project: Teiid
Issue Type: Bug
Components: Documentation
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Productization changes to use xref for links do not render correctly with the usual gitbook build. They produce an invalid anchor link in the current doc.
There are also some pages that look like they weren't updated to the new page names. Running a regex like :[^/A-Z:]*[A-Z][^/:]*.adoc over the reference guide shows at least 86 now invalid links. [~broldan] do you have a master list of old to new doc file names that could script getting those updated?
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-3647) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-3647?focusedWorklogId=12451527&pag... ]
Steven Hawkins logged work on TEIID-3647:
-----------------------------------------
Author: Steven Hawkins
Created on: 18/Jun/20 8:51 AM
Start Date: 18/Jun/20 8:49 AM
Worklog Time Spent: 4 hours, 30 minutes
Work Description: Time spent porting the implementation to wildfly and aligning the dependencies.
Issue Time Tracking
-------------------
Remaining Estimate: 1 hour, 30 minutes (was: 6 hours)
Time Spent: 4 hours, 30 minutes
Worklog Id: (was: 12451527)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIID-3647
> URL: https://issues.redhat.com/browse/TEIID-3647
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 6 hours
> Time Spent: 4 hours, 30 minutes
> Remaining Estimate: 1 hour, 30 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIIDSB-206) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-206?focusedWorklogId=12451526&pa... ]
Steven Hawkins logged work on TEIIDSB-206:
------------------------------------------
Author: Steven Hawkins
Created on: 18/Jun/20 8:49 AM
Start Date: 18/Jun/20 8:48 AM
Worklog Time Spent: 1 hour
Work Description: Additional time walking through Aditya's work.
Issue Time Tracking
-------------------
Remaining Estimate: 2 days, 30 minutes (was: 2 days, 1 hour, 30 minutes)
Time Spent: 3 hours, 30 minutes (was: 2 hours, 30 minutes)
Worklog Id: (was: 12451526)
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIIDSB-206
> URL: https://issues.redhat.com/browse/TEIIDSB-206
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Components: datasource
> Reporter: Van Halbert
> Assignee: Aditya Manglam Sharma
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 2 days, 4 hours
> Time Spent: 3 hours, 30 minutes
> Remaining Estimate: 2 days, 30 minutes
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months
[JBoss JIRA] (TEIID-3647) Create native connector to interact with HDFS as a datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-3647?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-3647:
----------------------------------
Original Estimate: 6 hours
Remaining Estimate: 6 hours
Story Points: 1
> Create native connector to interact with HDFS as a datasource
> -------------------------------------------------------------
>
> Key: TEIID-3647
> URL: https://issues.redhat.com/browse/TEIID-3647
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 6 hours
> Remaining Estimate: 6 hours
>
> Would like to have native connector
> To interact with HDFS as a datasource from a DV perspective there are at least two options:
> 1 - HttpFS
> 2 - WebHDFS
> Both options seems to be in practise very slow. I heard 2x slower.
> It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 5 months