[teiid-issues] [JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.

[Steven Hawkins (Jira)]

    [ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381564#comment-14381564 ] 

Steven Hawkins commented on TEIID-6015:
---------------------------------------

> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.

 

If you are using the amazon-s3 translator that uses a ws source.

 

If you are using the s3 source, then you would not use the amazon-s3 translator, but rather the file or excel translators.

 

Note that the s3 source does not support oath2 authentication, just the s3 secret/access keys.

 

>  how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not

 

That was already covered in TEIID-6011

 

> how to create VDB and datasource from the above configuration.

 

I'm not sure what you mean be datasource.  From what you have above, you already have an s3 source.

 

In an xml vdb you would do something like:
{code:java}
...
<!-- create a source model/schema using the s3DS via the file translator -->
<model name="s3-source">
  <source name="s3" translator-name="file" connection-jndi-name="java:/s3DS" />
</model>
<!-- create some views and other interesting stuff -->
<model name="s3-view" type="virtual">
  <metadata type="ddl"><![CDATA[
CREATE VIEW stock_price(symbol string, price double, PRIMARY KEY(symbol)) AS        SELECT tt.symbol, tt.price, FROM (EXEC "s3-source".getTextFiles('sample_data.csv')) AS f, TEXTTABLE(f.file COLUMNS symbol string, price double  HEADER) AS tt;
  ]]></metadata>
</model>
... 
{code}
 

If you are curious for Teiid on openshift in yaml/crd form, see [https://github.com/teiid/teiid-operator/blob/master/deploy/crs/vdb_for_s3.yaml]

 

 

 

 

 

 

 

 

> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
>                 Key: TEIID-6015
>                 URL: https://issues.redhat.com/browse/TEIID-6015
>             Project: Teiid
>          Issue Type: Feature Request
>            Reporter: Nayan Bija
>            Assignee: Steven Hawkins
>            Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940  ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket,  we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>  
> <security-domain name="oauth2-security">
>  <authentication>
>  <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
>  <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
>  <module-option name="client-secret" value="xxx"/>
>  <module-option name="refresh-token" value="xx"/>
>  <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
>  </login-module>
>  </authentication>
>  </security-domain>
>  </security-domains>
>  
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
>  <resource-adapter id="amazon-s3-custom">
>  <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/> 
>  <transaction-support>NoTransaction</transaction-support>
>  <connection-definitions> 
>  <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory"  jndi-name="java:/s3DS"  enabled="true"  use-java-context="true"  pool-name="teiid-s3-ds"> 
>     <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
>     <config-property name="bucket">abc</config-property>
>     <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property> 
>     <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
>  </resource-adapter>
>  
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>  
>  
>  
>  
>  
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v7.13.8#713008)