12:53 p.m.
[
https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi...
]
Steven Hawkins commented on TEIID-6015:
---------------------------------------
and we are using Amazon-s3 translator to read the csv file from the
GCP bucket.
If you are using the amazon-s3 translator that uses a ws source.
If you are using the s3 source, then you would not use the amazon-s3 translator, but
rather the file or excel translators.
Note that the s3 source does not support oath2 authentication, just the s3 secret/access
keys.
how to test the above configuration from the Teiid console whether
it is connected to GCP bucket or not
That was already covered in TEIID-6011
how to create VDB and datasource from the above configuration.
I'm not sure what you mean be datasource. From what you have above, you already have
an s3 source.
In an xml vdb you would do something like:
{code:java}
...
<!-- create a source model/schema using the s3DS via the file translator -->
<model name="s3-source">
<source name="s3"
translator-name="file" connection-jndi-name="java:/s3DS" />
</model>
<!-- create some views and other interesting stuff -->
<model name="s3-view" type="virtual">
<metadata type="ddl"><![CDATA[
CREATE VIEW stock_price(symbol string, price double, PRIMARY KEY(symbol)) AS SELECT
tt.symbol, tt.price, FROM (EXEC
"s3-source".getTextFiles('sample_data.csv')) AS f, TEXTTABLE(f.file
COLUMNS symbol string, price double HEADER) AS tt;
]]></metadata>
</model>
...
{code}
If you are curious for Teiid on openshift in yaml/crd form,
see [https://github.com/teiid/teiid-operator/blob/master/deploy/crs/vdb_f...
GCP Storage(bucket) connection with Teiid.
------------------------------------------
Key: TEIID-6015
URL: https://issues.redhat.com/browse/TEIID-6015
Project: Teiid
Issue Type: Feature Request
Reporter: Nayan Bija
Assignee: Steven Hawkins
Priority: Major
We need to read CSV file from the GCP storage file using SQL query.
we are following the steps mentioned in - TEIID-5940 ticket.
As suggested by [~shawkins] we have updated or Teiid to 15.x version.
and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
As suggested in the ticket, we have configured oauth2 in GCP and configured the secret
key and refresh token in security deomain. .
<security-domain name="oauth2-security">
<authentication>
<login-module code="org.teiid.jboss.oauth.OAuth20LoginModule"
flag="required" module="org.jboss.teiid.web.cxf">
<module-option name="client-id"
value="xxx.apps.googleusercontent.com"/>
<module-option name="client-secret" value="xxx"/>
<module-option name="refresh-token" value="xx"/>
<module-option name="access-token-uri"
value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
we have also created resource adaptor.
<resource-adapter id="amazon-s3-custom">
<resource-adapter id="amazon-s3-custom">
<module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition
class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory"
jndi-name="java:/s3DS" enabled="true"
use-java-context="true" pool-name="teiid-s3-ds">
<config-property
name="endpoint">[https://storage.googleapis.com|https://stor...
<config-property name="bucket">abc</config-property>
<config-property
name="accessKey">xxxx.apps.googleusercontent.com</config-property>
<config-property name="secretKey">xxxx</config-property>
</connection-definition> </connection-definitions>
</resource-adapter>
Could you please review the changes and suggest to us if we are missing anything.
and also let us inform
1. how to test the above configuration from the Teiid console whether it is connected to
GCP bucket or not
2. how to create VDB and datasource from the above configuration.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
2068
days inactive
2068
days old
0 comments
1 participants
participants (1)
-
Steven Hawkins (Jira)