[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Nayan Bija (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Nayan Bija commented on TEIID-6015:
-----------------------------------
Ok will try s3 then.
As you said we can't use oauth2 authentication so what should we need to provide in s3 secret/access keys.
As we have created the credentials for auth2 in gcp so we are having client id and client secret id of GCP.
So for S3 translator can we use
Client Id as access key and S3 secret as client-secret key of gcp.
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940 ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-6015:
---------------------------------------
I would guess s3 based upon their documentation, but I have only directly tested that against minio, ceph, and aws s3.
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940 ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Nayan Bija (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Nayan Bija commented on TEIID-6015:
-----------------------------------
To read CSV from GCP Bucket, which translator should we used?
S3 or Amazon-3?
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940 ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-6015:
---------------------------------------
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
If you are using the amazon-s3 translator that uses a ws source.
If you are using the s3 source, then you would not use the amazon-s3 translator, but rather the file or excel translators.
Note that the s3 source does not support oath2 authentication, just the s3 secret/access keys.
> how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
That was already covered in TEIID-6011
> how to create VDB and datasource from the above configuration.
I'm not sure what you mean be datasource. From what you have above, you already have an s3 source.
In an xml vdb you would do something like:
{code:java}
...
<!-- create a source model/schema using the s3DS via the file translator -->
<model name="s3-source">
<source name="s3" translator-name="file" connection-jndi-name="java:/s3DS" />
</model>
<!-- create some views and other interesting stuff -->
<model name="s3-view" type="virtual">
<metadata type="ddl"><![CDATA[
CREATE VIEW stock_price(symbol string, price double, PRIMARY KEY(symbol)) AS SELECT tt.symbol, tt.price, FROM (EXEC "s3-source".getTextFiles('sample_data.csv')) AS f, TEXTTABLE(f.file COLUMNS symbol string, price double HEADER) AS tt;
]]></metadata>
</model>
...
{code}
If you are curious for Teiid on openshift in yaml/crd form, see [https://github.com/teiid/teiid-operator/blob/master/deploy/crs/vdb_f...
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940 ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Steven Hawkins resolved TEIID-6015.
-----------------------------------
Resolution: Explained
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940 ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6014) Connection Pooling in SQL server XA data source
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-6014?page=com.atlassian.jira.plugi... ]
Steven Hawkins resolved TEIID-6014.
-----------------------------------
Resolution: Explained
> Initially 4 connection threads will be active.
Only if the prefill option is specified.
> Whenever we are going to use these connections one by one Available count will decrement.
I think this requires statistics-enabled to be true.
The various methods of adding an xa dataource - adminapi, directly editing the xml, using the cli, or even the higher level cli xa-datasource-add call - all have a variety of defaults and assumptions so you should confirm via the cli the ultimate configuration is as you expect:
/subsystem=datasources/xa-data-source=XXXX:read-resource
> Connection Pooling in SQL server XA data source
> -----------------------------------------------
>
> Key: TEIID-6014
> URL: https://issues.redhat.com/browse/TEIID-6014
> Project: Teiid
> Issue Type: Bug
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
>
> We are implementing connection pooling in the case of XA data source Sql Server.
>
> As per our understanding of the connection pooling mechanism in the teiid, we are referring below parameter in the Teiid Admin console.
> Below are the parameters mentioned in the Data Source Metrics
> i) Available count
> ii) Active count
> iii) Max used
>
> While deploying the XA data source for SQL server, we have configured the min pool size = 4 and max pool size = 8 and below are our understanding and how ideally it should work if we have implemented the connection pooling. (Please correct our observation)
> # Initially 4 connection threads will be active.
> # Whenever we are going to use these connections one by one Available count will decrement.
> # When all 8 connections will have used up, we will wait till connection available back to pool.
>
> Below are the actual observations.
> # No changes identified on Admin console.
> # On DB level threads are increasing irrespective of max pool size.
> # connections are increasing till 12 connections even though max size is 8
>
> Could you please shed more light on below points:
> # How SQL server XA datasource connection pooling is working.
> # In case of My sql XA datasource its working as expected.
> Attaching screenshot for Admin console statistics and DB level thread status.
> Please let us know are we missing any configuration.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Nayan Bija (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Nayan Bija updated TEIID-6015:
------------------------------
Description:
We need to read CSV file from the GCP storage file using SQL query.
we are following the steps mentioned in - TEIID-5940 ticket.
As suggested by [~shawkins] we have updated or Teiid to 15.x version.
and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
<security-domain name="oauth2-security">
<authentication>
<login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
<module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
<module-option name="client-secret" value="xxx"/>
<module-option name="refresh-token" value="xx"/>
<module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
we have also created resource adaptor.
<resource-adapter id="amazon-s3-custom">
<resource-adapter id="amazon-s3-custom">
<module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
<config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
<config-property name="bucket">abc</config-property>
<config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
<config-property name="secretKey">xxxx</config-property>
</connection-definition> </connection-definitions>
</resource-adapter>
Could you please review the changes and suggest to us if we are missing anything.
and also let us inform
1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
2. how to create VDB and datasource from the above configuration.
was:
We need to read CSV file from GCP storage file using SQL query.
we are following the steps maitined in - TEIID-5940 ticket.
As suggest by [~shawkins] we have updated or Teiid to 15.x version.
and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
As suggested in the ticket, we have configured oauth2 in GCP and configured secret key and refresh token in security deomain. .
<security-domain name="oauth2-security">
<authentication>
<login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
<module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
<module-option name="client-secret" value="xxx"/>
<module-option name="refresh-token" value="xx"/>
<module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
we have also created resource adaptor.
<resource-adapter id="amazon-s3-custom">
<resource-adapter id="amazon-s3-custom">
<module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
<config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
<config-property name="bucket">abc</config-property>
<config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
<config-property name="secretKey">xxxx</config-property>
</connection-definition> </connection-definitions>
</resource-adapter>
Could you please review the changes and suggest us if we are missing anything.
and also let us inform
1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
2. how to create VDB and datasource from above configuration.
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from the GCP storage file using SQL query.
> we are following the steps mentioned in - TEIID-5940 ticket.
> As suggested by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured the secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest to us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from the above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Nayan Bija (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Nayan Bija updated TEIID-6015:
------------------------------
Description:
We need to read CSV file from GCP storage file using SQL query.
we are following the steps maitined in - TEIID-5940 ticket.
As suggest by [~shawkins] we have updated or Teiid to 15.x version.
and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
As suggested in the ticket, we have configured oauth2 in GCP and configured secret key and refresh token in security deomain. .
<security-domain name="oauth2-security">
<authentication>
<login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
<module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
<module-option name="client-secret" value="xxx"/>
<module-option name="refresh-token" value="xx"/>
<module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
we have also created resource adaptor.
<resource-adapter id="amazon-s3-custom">
<resource-adapter id="amazon-s3-custom">
<module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
<config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
<config-property name="bucket">abc</config-property>
<config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
<config-property name="secretKey">xxxx</config-property>
</connection-definition> </connection-definitions>
</resource-adapter>
Could you please review the changes and suggest us if we are missing anything.
and also let us inform
1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
2. how to create VDB and datasource from above configuration.
was:
We need to read GCP storage CSV file data using SQL query.
we are following the steps maitined in - TEIID-5940 ticket.
As suggest by [~shawkins] we have updated or Teiid to 15.x version.
As suggested in the ticket, we have configured oauth2 in GCP and configured secret key and refresh token in security deomain. .
<security-domain name="oauth2-security">
<authentication>
<login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
<module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
<module-option name="client-secret" value="xxx"/>
<module-option name="refresh-token" value="xx"/>
<module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
we have also created resource adaptor.
<resource-adapter id="amazon-s3-custom">
<resource-adapter id="amazon-s3-custom">
<module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
<config-property name="endpoint">https://storage.googleapis.com</config-property>
<config-property name="bucket">abc</config-property>
<config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
<config-property name="secretKey">xxxx</config-property>
</connection-definition> </connection-definitions>
</resource-adapter>
Could you please review the changes and suggest us if we are missing anything.
and also let us inform
1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
2. how to create VDB and datasource from above configuration.
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read CSV file from GCP storage file using SQL query.
> we are following the steps maitined in - TEIID-5940 ticket.
> As suggest by [~shawkins] we have updated or Teiid to 15.x version.
> and we are using Amazon-s3 translator to read the csv file from the GCP bucket.
> As suggested in the ticket, we have configured oauth2 in GCP and configured secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">[https://storage.googleapis.com|https://storage.googleapis.com/]</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months
[JBoss JIRA] (TEIID-6015) GCP Storage(bucket) connection with Teiid.
by Nayan Bija (Jira)
[ https://issues.redhat.com/browse/TEIID-6015?page=com.atlassian.jira.plugi... ]
Nayan Bija updated TEIID-6015:
------------------------------
Description:
We need to read GCP storage CSV file data using SQL query.
we are following the steps maitined in - TEIID-5940 ticket.
As suggest by [~shawkins] we have updated or Teiid to 15.x version.
As suggested in the ticket, we have configured oauth2 in GCP and configured secret key and refresh token in security deomain. .
<security-domain name="oauth2-security">
<authentication>
<login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
<module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
<module-option name="client-secret" value="xxx"/>
<module-option name="refresh-token" value="xx"/>
<module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
we have also created resource adaptor.
<resource-adapter id="amazon-s3-custom">
<resource-adapter id="amazon-s3-custom">
<module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
<config-property name="endpoint">https://storage.googleapis.com</config-property>
<config-property name="bucket">abc</config-property>
<config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
<config-property name="secretKey">xxxx</config-property>
</connection-definition> </connection-definitions>
</resource-adapter>
Could you please review the changes and suggest us if we are missing anything.
and also let us inform
1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
2. how to create VDB and datasource from above configuration.
> GCP Storage(bucket) connection with Teiid.
> ------------------------------------------
>
> Key: TEIID-6015
> URL: https://issues.redhat.com/browse/TEIID-6015
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Nayan Bija
> Assignee: Steven Hawkins
> Priority: Major
>
> We need to read GCP storage CSV file data using SQL query.
> we are following the steps maitined in - TEIID-5940 ticket.
> As suggest by [~shawkins] we have updated or Teiid to 15.x version.
> As suggested in the ticket, we have configured oauth2 in GCP and configured secret key and refresh token in security deomain. .
>
> <security-domain name="oauth2-security">
> <authentication>
> <login-module code="org.teiid.jboss.oauth.OAuth20LoginModule" flag="required" module="org.jboss.teiid.web.cxf">
> <module-option name="client-id" value="xxx.apps.googleusercontent.com"/>
> <module-option name="client-secret" value="xxx"/>
> <module-option name="refresh-token" value="xx"/>
> <module-option name="access-token-uri" value="https://www.googleapis.com/oauth2/v3/token"/>
> </login-module>
> </authentication>
> </security-domain>
> </security-domains>
>
> we have also created resource adaptor.
> <resource-adapter id="amazon-s3-custom">
> <resource-adapter id="amazon-s3-custom">
> <module slot="main" id="org.jboss.teiid.resource-adapter.s3"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.s3.S3ManagedConnectionFactory" jndi-name="java:/s3DS" enabled="true" use-java-context="true" pool-name="teiid-s3-ds">
> <config-property name="endpoint">https://storage.googleapis.com</config-property>
> <config-property name="bucket">abc</config-property>
> <config-property name="accessKey">xxxx.apps.googleusercontent.com</config-property>
> <config-property name="secretKey">xxxx</config-property>
> </connection-definition> </connection-definitions>
> </resource-adapter>
>
> Could you please review the changes and suggest us if we are missing anything.
> and also let us inform
> 1. how to test the above configuration from the Teiid console whether it is connected to GCP bucket or not
> 2. how to create VDB and datasource from above configuration.
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 9 months