[
https://issues.redhat.com/browse/TEIIDSB-211?page=com.atlassian.jira.plug...
]
Steven Hawkins commented on TEIIDSB-211:
----------------------------------------
The amazon sdk has examples here:
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3.html
The sdk will work against other providers, such as ceph or minio:
https://docs.min.io/docs/how-to-use-aws-sdk-for-java-with-minio-server.html
minio is great for testing:
https://docs.min.io/docs/minio-docker-quickstart-guide.html
you can also use podman instead of docker: podman run -p 9000:9000 minio/minio server
/data
see
https://podman.io/ and
https://podman.io/getting-started/installation.html
What we'll want is another source that supports the FileConnection/VirtualFile
interfaces. There is additional support that can be added later, but that is the initial
target. It looks like amazon produces an sdk release nearly every week, so we
shouldn't worry too much about the specific version initially.
The source will need details like, bucket, region, access key, encryption support etc. as
configuration properties. You can probably just use the property set from the current
Teiid S3ExecutionFactory, but also include an endPoint property - which can default to
aws.
create an s3 file source
-------------------------
Key: TEIIDSB-211
URL:
https://issues.redhat.com/browse/TEIIDSB-211
Project: Teiid Spring Boot
Issue Type: Feature Request
Components: datasource
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Priority: Major
Fix For: 1.6.0
The existing s3 support is implemented as a translator / ws source combo. We instead
need this to be just a source to allow for translators to utilize it (excel, parquet,
avro).
As an alternative to our existing support we should evaluate utilizing the sdk rather
than providing our own processing logic. More than likely this will be a quicker path to
things like s3 select support. However ceph seems to lag in s3 support (see TEIID-5935
and no s3 select support) so we'd have to compensate, make the metadata and/or the
capabilities specific to the connection type.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)