[JBoss JIRA] (TEIID-5927) S3 translator can not support minIO endpoint
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5927?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5927:
---------------------------------------
> I believe I handled both implicit/ddl cases with TEIIDSB-136, but can double check and write a unit test for it. If not yes.
It was just implicit. The assumption being that ddl would be used to set translator properties.
> S3 translator can not support minIO endpoint
> --------------------------------------------
>
> Key: TEIID-5927
> URL: https://issues.redhat.com/browse/TEIID-5927
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Rui Wang
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
> Original Estimate: 6 hours
> Remaining Estimate: 6 hours
>
> When I try to config my on own minIO storage to teiied RA, get following error, seems it tied directly with amazon endpoints, could you pls takes an enhancement to make it generic so it can be used with minio stores. Thanks a lot.
> <resource-adapter id="webservice">
> <module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/wsDS" enabled="true" use-java-context="true" pool-name="wsDS">
> <config-property name="EndPoint">
> http://myMinioIp:9091/minio/bucket1/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
>
>
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:783)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:581)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:590)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:351)
> at com.ibm.vdb.TeiidFile.showVdbInfo(TeiidFile.java:181)
> at com.ibm.vdb.TeiidFile.main(TeiidFile.java:205)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:397)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:157)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:401)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.relational.SourceState.prefetch(SourceState.java:207)
> at org.teiid.query.processor.relational.SourceState.rowCountLE(SourceState.java:154)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:252)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:147)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:110)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:492)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:124)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$2.run(ThreadReuseExecutor.java:212)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.translator.amazon.s3.S3ProcedureExecution.execute(S3ProcedureExecution.java:114)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:402)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:228)
> at com.sun.proxy.$Proxy43.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:59)
> ... 6 more
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5927) S3 translator can not support minIO endpoint
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIID-5927?page=com.atlassian.jira.plugi... ]
Ramesh Reddy commented on TEIID-5927:
-------------------------------------
TEIID-5936 sounds good, we can replace the current one. Since security stuff is handled in the library and specific we do not have to spend more time on spring security or cxf security.
I believe I handled both implicit/ddl cases with TEIIDSB-136, but can double check and write a unit test for it. If not yes.
> S3 translator can not support minIO endpoint
> --------------------------------------------
>
> Key: TEIID-5927
> URL: https://issues.redhat.com/browse/TEIID-5927
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Rui Wang
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
> Original Estimate: 6 hours
> Remaining Estimate: 6 hours
>
> When I try to config my on own minIO storage to teiied RA, get following error, seems it tied directly with amazon endpoints, could you pls takes an enhancement to make it generic so it can be used with minio stores. Thanks a lot.
> <resource-adapter id="webservice">
> <module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/wsDS" enabled="true" use-java-context="true" pool-name="wsDS">
> <config-property name="EndPoint">
> http://myMinioIp:9091/minio/bucket1/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
>
>
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:783)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:581)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:590)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:351)
> at com.ibm.vdb.TeiidFile.showVdbInfo(TeiidFile.java:181)
> at com.ibm.vdb.TeiidFile.main(TeiidFile.java:205)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:397)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:157)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:401)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.relational.SourceState.prefetch(SourceState.java:207)
> at org.teiid.query.processor.relational.SourceState.rowCountLE(SourceState.java:154)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:252)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:147)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:110)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:492)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:124)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$2.run(ThreadReuseExecutor.java:212)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.translator.amazon.s3.S3ProcedureExecution.execute(S3ProcedureExecution.java:114)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:402)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:228)
> at com.sun.proxy.$Proxy43.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:59)
> ... 6 more
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5936) evaluate aws sdk for java for s3 support
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5936?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5936:
---------------------------------------
Adding a new source / translator would also give us allow us to use a more generic name - currently this support is called amazon-s3, which we could shorten to just s3.
> evaluate aws sdk for java for s3 support
> ----------------------------------------
>
> Key: TEIID-5936
> URL: https://issues.redhat.com/browse/TEIID-5936
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
>
> As an alternative to our existing support we should evaluate utilizing the sdk rather than providing our own processing logic. More than likely this will be a quicker path to things like s3 select support. However ceph seems to lag in s3 support (see TEIID-5935 and no s3 select support) so we'd have to compensate, make the metadata and/or the capabilities specific to the connection type.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5935) S3 support for ceph
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5935?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5935:
----------------------------------
Parent: (was: TEIID-5927)
Issue Type: Feature Request (was: Sub-task)
> S3 support for ceph
> -------------------
>
> Key: TEIID-5935
> URL: https://issues.redhat.com/browse/TEIID-5935
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
>
> We need to validate ceph support against the changes in TEIID-5927. The most immediate issue appears to be that ceph does not support the v2 listing of bucket contents and would need to use an older list method.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5927) S3 translator can not support minIO endpoint
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5927?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5927:
---------------------------------------
The changes for minio now seem to work.
[~rareddy] There is a lot of connectivity details in translator properties - region, access key, and secret key - which are also fully exposed by the procedures. For the paradigm of having a split between translator and source, this makes environmental migration difficult. For an explicit vdb with Teiid Spring Boot / crds we can still have an issue as TEIIDSB-136 only applies to implicit vdbs - that is you currently cannot set a translator property anything other than ddl. If we go down the path of TEIID-5936, then we'll create an s3 source which will better encapsulate things. We may also consider if / how we want users to update execution properties via a crd / teiid spring boot.
> S3 translator can not support minIO endpoint
> --------------------------------------------
>
> Key: TEIID-5927
> URL: https://issues.redhat.com/browse/TEIID-5927
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Rui Wang
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
> Original Estimate: 6 hours
> Remaining Estimate: 6 hours
>
> When I try to config my on own minIO storage to teiied RA, get following error, seems it tied directly with amazon endpoints, could you pls takes an enhancement to make it generic so it can be used with minio stores. Thanks a lot.
> <resource-adapter id="webservice">
> <module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/wsDS" enabled="true" use-java-context="true" pool-name="wsDS">
> <config-property name="EndPoint">
> http://myMinioIp:9091/minio/bucket1/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
>
>
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:783)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:581)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:590)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:351)
> at com.ibm.vdb.TeiidFile.showVdbInfo(TeiidFile.java:181)
> at com.ibm.vdb.TeiidFile.main(TeiidFile.java:205)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:397)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:157)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:401)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.relational.SourceState.prefetch(SourceState.java:207)
> at org.teiid.query.processor.relational.SourceState.rowCountLE(SourceState.java:154)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:252)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:147)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:110)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:492)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:124)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$2.run(ThreadReuseExecutor.java:212)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.translator.amazon.s3.S3ProcedureExecution.execute(S3ProcedureExecution.java:114)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:402)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:228)
> at com.sun.proxy.$Proxy43.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:59)
> ... 6 more
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-190) REST with request payload throws an error
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-190?page=com.atlassian.jira.plug... ]
Steven Hawkins reassigned TEIIDSB-190:
--------------------------------------
Issue Type: Bug (was: Enhancement)
Fix Version/s: 1.5.0
Sprint: DV Sprint 62
Assignee: Ramesh Reddy
> REST with request payload throws an error
> -----------------------------------------
>
> Key: TEIIDSB-190
> URL: https://issues.redhat.com/browse/TEIIDSB-190
> Project: Teiid Spring Boot
> Issue Type: Bug
> Components: core
> Reporter: Sadhana Nandakumar
> Assignee: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
>
> TEIID spring boot is expecting the payload to be a bean, and hence throws an error. POST without request body works fine without any issues.
> CREATE VIEW StockPrice (
> symbol string PRIMARY KEY,
> price double
> ) AS
> SELECT p.symbol, y.price
> FROM accounts.PRODUCT as p, TABLE(call invokeHttp(action=>'POST', endpoint=>QUERYSTRING('predict'), headers=>jsonObject('application/json' as "Content-Type"), request=>jsonObject('PLATINUM' as "customerClass"))) as x,
> JSONTABLE(JSONPARSE(x.result,true), '$' COLUMNS price double path '@.data.segment') as y
> No errors on build, when I query the odata endpoint, I see the following error.
> <error xmlns="http://docs.oasis-open.org/odata/ns/metadata">
> <code>TEIID30504</code>
> <message>
> TEIID30504 quotesvc: No bean named '{"customerClass":"PLATINUM"}' available
> </message>
> </error>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5927) S3 translator can not support minIO endpoint
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5927?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5927:
---------------------------------------
My initial efforts didn't quite work out. There is something that minio doesn't like about the request signature. It's not clear yet after a couple of debug iterations what is different between the signature generated by the sdk and our code which seems to be based upon an older example from amazon.
> S3 translator can not support minIO endpoint
> --------------------------------------------
>
> Key: TEIID-5927
> URL: https://issues.redhat.com/browse/TEIID-5927
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Rui Wang
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
> Original Estimate: 6 hours
> Remaining Estimate: 6 hours
>
> When I try to config my on own minIO storage to teiied RA, get following error, seems it tied directly with amazon endpoints, could you pls takes an enhancement to make it generic so it can be used with minio stores. Thanks a lot.
> <resource-adapter id="webservice">
> <module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/wsDS" enabled="true" use-java-context="true" pool-name="wsDS">
> <config-property name="EndPoint">
> http://myMinioIp:9091/minio/bucket1/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
>
>
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:783)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:581)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:590)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:351)
> at com.ibm.vdb.TeiidFile.showVdbInfo(TeiidFile.java:181)
> at com.ibm.vdb.TeiidFile.main(TeiidFile.java:205)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:397)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:157)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:401)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.relational.SourceState.prefetch(SourceState.java:207)
> at org.teiid.query.processor.relational.SourceState.rowCountLE(SourceState.java:154)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:252)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:147)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:110)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:492)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:124)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$2.run(ThreadReuseExecutor.java:212)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.translator.amazon.s3.S3ProcedureExecution.execute(S3ProcedureExecution.java:114)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:402)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:228)
> at com.sun.proxy.$Proxy43.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:59)
> ... 6 more
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-186) Fix to use only use alias name from ExternalSource
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-186?focusedWorklogId=12450824&pa... ]
Ramesh Reddy logged work on TEIIDSB-186:
----------------------------------------
Author: Ramesh Reddy
Created on: 14/Apr/20 7:03 PM
Start Date: 14/Apr/20 7:03 PM
Worklog Time Spent: 5 hours
Work Description: Steve's suggested changes
Issue Time Tracking
-------------------
Time Spent: 3 days, 5 hours (was: 3 days)
Worklog Id: (was: 12450824)
> Fix to use only use alias name from ExternalSource
> --------------------------------------------------
>
> Key: TEIIDSB-186
> URL: https://issues.redhat.com/browse/TEIIDSB-186
> Project: Teiid Spring Boot
> Issue Type: Quality Risk
> Components: core
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
> Time Spent: 3 days, 5 hours
> Remaining Estimate: 0 minutes
>
> As the code progressed the usage of translator name and alias name from ExternalSource class is interchangeably used. This is leading to confusion in terms of development and documents.
> Need to fix all the places where the translator name used to use alias name with Spring Boot engine.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIIDSB-190) REST with request payload throws an error
by Sadhana Nandakumar (Jira)
Sadhana Nandakumar created TEIIDSB-190:
------------------------------------------
Summary: REST with request payload throws an error
Key: TEIIDSB-190
URL: https://issues.redhat.com/browse/TEIIDSB-190
Project: Teiid Spring Boot
Issue Type: Enhancement
Components: core
Reporter: Sadhana Nandakumar
TEIID spring boot is expecting the payload to be a bean, and hence throws an error. POST without request body works fine without any issues.
CREATE VIEW StockPrice (
symbol string PRIMARY KEY,
price double
) AS
SELECT p.symbol, y.price
FROM accounts.PRODUCT as p, TABLE(call invokeHttp(action=>'POST', endpoint=>QUERYSTRING('predict'), headers=>jsonObject('application/json' as "Content-Type"), request=>jsonObject('PLATINUM' as "customerClass"))) as x,
JSONTABLE(JSONPARSE(x.result,true), '$' COLUMNS price double path '@.data.segment') as y
No errors on build, when I query the odata endpoint, I see the following error.
<error xmlns="http://docs.oasis-open.org/odata/ns/metadata">
<code>TEIID30504</code>
<message>
TEIID30504 quotesvc: No bean named '{"customerClass":"PLATINUM"}' available
</message>
</error>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (TEIID-5936) evaluate aws sdk for java for s3 support
by Steven Hawkins (Jira)
Steven Hawkins created TEIID-5936:
-------------------------------------
Summary: evaluate aws sdk for java for s3 support
Key: TEIID-5936
URL: https://issues.redhat.com/browse/TEIID-5936
Project: Teiid
Issue Type: Feature Request
Components: Misc. Connectors
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 14.0
As an alternative to our existing support we should evaluate utilizing the sdk rather than providing our own processing logic. More than likely this will be a quicker path to things like s3 select support. However ceph seems to lag in s3 support (see TEIID-5935 and no s3 select support) so we'd have to compensate, make the metadata and/or the capabilities specific to the connection type.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months