[
https://issues.redhat.com/browse/TEIIDSB-206?page=com.atlassian.jira.plug...
]
Steven Hawkins commented on TEIIDSB-206:
----------------------------------------
File and HDFS and S3 all becoming synonymous to me, I am a little
confused to see `file-hdfs` kind of naming.
They are all virtual file sources, but obviously hit different things. I know we talked
about this in the past, but I deferred on a strategy.
To explain this more, what we currently have:
CREATE SERVER x1 FOREIGN DATA WRAPPER file; -- currently means file source with a file
translator
CREATE SERVER x2 FOREIGN DATA WRAPPER excel; -- currently means file source with an excel
translator
CREATE SERVER x3 FOREIGN DATA WRAPPER ftp; -- currently means an ftp source with an ftp
translator, which is a bug because there is no ftp translator
What has just been added:
CREATE SERVER x1 FOREIGN DATA WRAPPER hdfs; -- currently means an ftp source with an ftp
translator, which is a bug because there is no hdfs translator
So obviously the last two need to point to file translator instead.
But the real issue is that this approach is baking in a single translator for a given
source name. excel can work against file, ftp, hdfs, and s3 once it's added.
I see that if you are going to design above `avro` as a language
feature then we do not need this?
The rationale for doing the hdfs and s3 work first was actually so that avro and parquet
support would make sense at a translator level.
If not we can see how we can abstract all these types under `file`
type and figure out dynamically as you suggest.
Option 1: use the create server type information. You can default an alias to a given
translator, but also specify it:
CREATE SERVER x2 FOREIGN DATA WRAPPER excel
could be fully expressed as: CREATE SERVER x2 TYPE file FOREIGN DATA WRAPPER excel
and would allow for the other combinations: CREATE SERVER x2 TYPE hdfs FOREIGN DATA
WRAPPER excel
The only downside is explicitly introducing the source type construct.
Option 2: explicit and verbose expansion of alias names / connection factories. create
excel-file, excel-ftp, excel-hdfs, avro-hdfs, etc.
Option 3: assume some kind of convention in the foreign data wrapper name.
Create native connector to interact with HDFS as a datasource
-------------------------------------------------------------
Key: TEIIDSB-206
URL:
https://issues.redhat.com/browse/TEIIDSB-206
Project: Teiid Spring Boot
Issue Type: Feature Request
Components: datasource
Reporter: Van Halbert
Assignee: Aditya Manglam Sharma
Priority: Major
Fix For: 1.6.0
Original Estimate: 2 days, 4 hours
Time Spent: 3 hours, 30 minutes
Remaining Estimate: 2 days, 30 minutes
Would like to have native connector
To interact with HDFS as a datasource from a DV perspective there are at least two
options:
1 - HttpFS
2 - WebHDFS
Both options seems to be in practise very slow. I heard 2x slower.
It would be nice if we can create a native HDFS connector.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)