[JBoss JIRA] (TEIIDSB-163) Provide pooling for BaseConnectionFactory sources
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-163?page=com.atlassian.jira.plug... ]
Ramesh Reddy updated TEIIDSB-163:
---------------------------------
Sprint: (was: DV Sprint 61)
> Provide pooling for BaseConnectionFactory sources
> -------------------------------------------------
>
> Key: TEIIDSB-163
> URL: https://issues.redhat.com/browse/TEIIDSB-163
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Components: datasource
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.5.0
>
>
> It will depend on the source and the cost of creating a connection, but in general we should provide pooling for connections rather than creating the for every operation. For example salesforce is a heavy weight connection.
> The pooling will need to be tied into spring security or jca like logic to properly manage connections by identity once we re-establish that logic.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 9 months
[JBoss JIRA] (TEIID-5928) Allow External and Internal materialization of multi-source to fail/update individually
by Rafael Sampaio (Jira)
Rafael Sampaio created TEIID-5928:
-------------------------------------
Summary: Allow External and Internal materialization of multi-source to fail/update individually
Key: TEIID-5928
URL: https://issues.redhat.com/browse/TEIID-5928
Project: Teiid
Issue Type: Enhancement
Components: Common
Affects Versions: 12.2.2
Reporter: Rafael Sampaio
Assignee: Steven Hawkins
Hi, all.
First of all thanks in advance for your attention, and efforts on such a great product.
Since I haven't found any docs or examples regarding my current usage (please if this already exists or there's a better way of doing this, disregard this request and advice) here goes my suggestion:
It would be nice to have a way to gracefully update/fail per source, if a view that has Materialization enabled and target a multi-source source model, because things can get, messy if I have to create a view per source and Materialize each one and only then create a unified (UNION) view.
Using this approach also obligates me to implement custom insert/update/delete/based on the discriminator column.
Thanks again.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 9 months
[JBoss JIRA] (TEIIDSB-187) Need to pull out the "property prefix" to ExternalSource
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-187?page=com.atlassian.jira.plug... ]
Ramesh Reddy updated TEIIDSB-187:
---------------------------------
Fix Version/s: 1.5.0
> Need to pull out the "property prefix" to ExternalSource
> --------------------------------------------------------
>
> Key: TEIIDSB-187
> URL: https://issues.redhat.com/browse/TEIIDSB-187
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: core
> Reporter: Ramesh Reddy
> Priority: Major
> Fix For: 1.5.0
>
>
> The prefix of the source like "org.spring.teiid.data.xxx" kind of property patterns are defined in the ConnectionFactory classes, this needs to be pulled into ExternalSource class, then this can be one central place everything about translator and source is managed.
> Consider externalizing ExternalSource information into a "json" configuration file, then this can be then used in Operator code load and enforce dependencies and pom file generation and property generation from a single location. Right now all this looks very hacky as it spread out across projects.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 9 months
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-23?page=com.atlassian.jira.plugi... ]
Ramesh Reddy resolved TEIIDSB-23.
---------------------------------
Fix Version/s: 1.5.0
Resolution: Done
I am marking this resolved, any remain issues can be dealt with in separate issues
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.redhat.com/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.5.0
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 9 months
[JBoss JIRA] (TEIID-5927) S3 translator can not support minIO endpoint
by Ramesh Reddy (Jira)
[ https://issues.redhat.com/browse/TEIID-5927?page=com.atlassian.jira.plugi... ]
Ramesh Reddy commented on TEIID-5927:
-------------------------------------
It is mentioned to me that Amazon libraries can be used with other compliant S3 implementations, I guess we need to see what is the connector is forcing it to be only used by the Amazon-S3
> S3 translator can not support minIO endpoint
> --------------------------------------------
>
> Key: TEIID-5927
> URL: https://issues.redhat.com/browse/TEIID-5927
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Rui Wang
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 14.0
>
>
> When I try to config my on own minIO storage to teiied RA, get following error, seems it tied directly with amazon endpoints, could you pls takes an enhancement to make it generic so it can be used with minio stores. Thanks a lot.
> <resource-adapter id="webservice">
> <module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/wsDS" enabled="true" use-java-context="true" pool-name="wsDS">
> <config-property name="EndPoint">
> http://myMinioIp:9091/minio/bucket1/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
>
>
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:131)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:67)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:783)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:581)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:131)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:36)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:281)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:333)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:423)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:590)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:420)
> at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:351)
> at com.ibm.vdb.TeiidFile.showVdbInfo(TeiidFile.java:181)
> at com.ibm.vdb.TeiidFile.main(TeiidFile.java:205)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 web-connector: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:397)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:157)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:401)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.relational.SourceState.prefetch(SourceState.java:207)
> at org.teiid.query.processor.relational.SourceState.rowCountLE(SourceState.java:154)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:252)
> at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:139)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:147)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:110)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:492)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:124)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$2.run(ThreadReuseExecutor.java:212)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Error reading the file https://s3.amazonaws.com/bucket1/price.csv; The response code returned 403. Details: <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>minioadmin</AWSAccessKeyId><RequestId>EB7421ECD8EF9B88</RequestId><HostId>m3fHokjXC5shrV+y+2vIHl7KNcr46RvoKRXuZ8TnU4fiZyi1XG6rT9DiTu6yRZfg328QGcr2w+A=</HostId></Error>
> at org.teiid.translator.amazon.s3.S3ProcedureExecution.execute(S3ProcedureExecution.java:114)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:402)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:228)
> at com.sun.proxy.$Proxy43.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:59)
> ... 6 more
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 9 months