[JBoss JIRA] (TEIID-5865) Error with pg arrays
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5865?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5865:
----------------------------------
Component/s: Query Engine
Description:
When I try the following query using customer-dv deployment against postgresql
https://rdbms-springboot-dv-demo.apps.cluster-london-966d.london-966d.ope...
I get the following error:
{"error":{"code":"TEIID30504","message":"TEIID30504 sampledb: TEIID31176 Could not convert source column v_0.c_1 AS c_3 to the expected runtime type object[][]"}}
This occurs with any pg array that is created with an expression:
select (col, col2)..
array_agg, etc.
was:
When I try the following query using customer-dv deployment against postgresql
https://rdbms-springboot-dv-demo.apps.cluster-london-966d.london-966d.ope...
I get the following error:
{"error":{"code":"TEIID30504","message":"TEIID30504 sampledb: TEIID31176 Could not convert source column v_0.c_1 AS c_3 to the expected runtime type object[][]"}}
Fix Version/s: 13.0.1
12.2.2
12.3.1
Sprint: DV Sprint 56
Summary: Error with pg arrays (was: Odata $expand does not work)
> Error with pg arrays
> --------------------
>
> Key: TEIID-5865
> URL: https://issues.redhat.com/browse/TEIID-5865
> Project: Teiid
> Issue Type: Bug
> Components: OData, Query Engine
> Affects Versions: 12.3
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.2.2, 12.3.1, 13.1, 13.0.1
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> When I try the following query using customer-dv deployment against postgresql
> https://rdbms-springboot-dv-demo.apps.cluster-london-966d.london-966d.ope...
> I get the following error:
> {"error":{"code":"TEIID30504","message":"TEIID30504 sampledb: TEIID31176 Could not convert source column v_0.c_1 AS c_3 to the expected runtime type object[][]"}}
> This occurs with any pg array that is created with an expression:
> select (col, col2)..
> array_agg, etc.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIID-5865) Odata $expand does not work
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5865?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5865:
---------------------------------------
Long story short - any array that we currently construct with pushdown sql using parens is actually a record type and is returned as pg object. This was done because using the explicit array syntax ARRAY[] only supports elements of the same type. However even in the one dimensional case the translator still isn't accounting for this. Since we do support mixed type arrays we need to use records and always correct the result.
> Odata $expand does not work
> ---------------------------
>
> Key: TEIID-5865
> URL: https://issues.redhat.com/browse/TEIID-5865
> Project: Teiid
> Issue Type: Bug
> Components: OData
> Affects Versions: 12.3
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 13.1
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> When I try the following query using customer-dv deployment against postgresql
> https://rdbms-springboot-dv-demo.apps.cluster-london-966d.london-966d.ope...
> I get the following error:
> {"error":{"code":"TEIID30504","message":"TEIID30504 sampledb: TEIID31176 Could not convert source column v_0.c_1 AS c_3 to the expected runtime type object[][]"}}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIID-5865) Odata $expand does not work
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5865?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5865:
---------------------------------------
The issue is that the SQL array returned by pg contains PGObject instances - not arrays. So there either needs to be additional translator logic to handle multi-dimensional array results, or array_agg at least for array values would need to be disabled. Since we don't currently have the type specific checks on aggregate functions I'll first see if the translator can handle converting the PGObject values to arrays.
> Odata $expand does not work
> ---------------------------
>
> Key: TEIID-5865
> URL: https://issues.redhat.com/browse/TEIID-5865
> Project: Teiid
> Issue Type: Bug
> Components: OData
> Affects Versions: 12.3
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 13.1
>
> Original Estimate: 5 hours
> Remaining Estimate: 5 hours
>
> When I try the following query using customer-dv deployment against postgresql
> https://rdbms-springboot-dv-demo.apps.cluster-london-966d.london-966d.ope...
> I get the following error:
> {"error":{"code":"TEIID30504","message":"TEIID30504 sampledb: TEIID31176 Could not convert source column v_0.c_1 AS c_3 to the expected runtime type object[][]"}}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIIDSB-51) Define expectations for runtime metadata
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-51?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIIDSB-51:
---------------------------------------
At this point our messaging will be that it's a best practice to use static metadata. Other than teiid-syndesis we won't provide an explicit tool that will extract metadata for a vdb, but it is at least somewhat straight-forward to capture source ddl.
For import foreign schema statements (dynamic metadata) we'll highlight the benefits and drawbacks:
- will refresh when the pod is started
- may increase the startup time (if and until we introduce metadata caching / storage which is discussed a little above)
- when dealing with things like system queries the behavior won't be guaranteed to be immutable - teiid ids can change, the number of objects can change, etc.
> Define expectations for runtime metadata
> ----------------------------------------
>
> Key: TEIIDSB-51
> URL: https://issues.redhat.com/browse/TEIIDSB-51
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.3.0
>
>
> At a high level there are potentially three modes for our ddl:
> 1. Interactive - there was some work toward this, but it needed to only be a "developer mode" as it would reload the vdb on each metadata change. There is a lot about the transactionality of DDL updates that our current simplistic metadata model does not support, which makes this hard to fully implement. We haven't talked about this much in relation to Teiid Spring Boot, but ideally there should be a way for developers to test incremental changes without full rebuilds.
> 2. Dynamic - the term is a bit dated, but this refers to any vdb that performs import from a metadata repository that has runtime dependencies. By design this allows for a simple vdb definition and options for caching or re-importing on reload. In a container environment metadata caching no longer makes sense unless we use a volume. Depending upon your viewpoint allowing for runtime import of metadata is potentially a point of mutability that is also not needed in a container environment.
> 3. Static - much like our old .index designer vdbs there are benefits to having the full metadata already available to the vdb/image. There is a lot of work that we do lazily at runtime that can be moved to an earlier phase - primarily generating the rest and odata layers, but it would also include the pg system table materializations. For Teiid Spring Boot or usage in containers in general what this would look like is a build phase for the vdb such that it could obtain appropriate metadata and initialize as much as possible statically (eventually for quarkus vm snapshotting). The only metadata that would not be fresh potentially would be the costing metadata - however that is problematic as is and will eventually need a progressive optimization strategy employing a persistent store.
> As of now we are currently focused just on #2. As we look toward operators, we should start thinking about #3.
> #1 would come into play if/when we start looking at local development options using vs code.
> We can treat this issue as an epic and a place to centralize discussion.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIIDSB-51) Define expectations for runtime metadata
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-51?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIIDSB-51:
----------------------------------
Component/s: documentation
> Define expectations for runtime metadata
> ----------------------------------------
>
> Key: TEIIDSB-51
> URL: https://issues.redhat.com/browse/TEIIDSB-51
> Project: Teiid Spring Boot
> Issue Type: Feature Request
> Components: documentation
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.3.0
>
>
> At a high level there are potentially three modes for our ddl:
> 1. Interactive - there was some work toward this, but it needed to only be a "developer mode" as it would reload the vdb on each metadata change. There is a lot about the transactionality of DDL updates that our current simplistic metadata model does not support, which makes this hard to fully implement. We haven't talked about this much in relation to Teiid Spring Boot, but ideally there should be a way for developers to test incremental changes without full rebuilds.
> 2. Dynamic - the term is a bit dated, but this refers to any vdb that performs import from a metadata repository that has runtime dependencies. By design this allows for a simple vdb definition and options for caching or re-importing on reload. In a container environment metadata caching no longer makes sense unless we use a volume. Depending upon your viewpoint allowing for runtime import of metadata is potentially a point of mutability that is also not needed in a container environment.
> 3. Static - much like our old .index designer vdbs there are benefits to having the full metadata already available to the vdb/image. There is a lot of work that we do lazily at runtime that can be moved to an earlier phase - primarily generating the rest and odata layers, but it would also include the pg system table materializations. For Teiid Spring Boot or usage in containers in general what this would look like is a build phase for the vdb such that it could obtain appropriate metadata and initialize as much as possible statically (eventually for quarkus vm snapshotting). The only metadata that would not be fresh potentially would be the costing metadata - however that is problematic as is and will eventually need a progressive optimization strategy employing a persistent store.
> As of now we are currently focused just on #2. As we look toward operators, we should start thinking about #3.
> #1 would come into play if/when we start looking at local development options using vs code.
> We can treat this issue as an epic and a place to centralize discussion.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-23?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIIDSB-23:
---------------------------------------
Linking to the metrics task - there we configure the s2i java image jolokia support to be a wide open http service and have implemented basic scraping in teiid-syndesis. There are of course a lot of of other options for jolokia - including exposing via actuator which is likely what anyone using spring boot directly would do.
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.redhat.com/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIIDSB-109) Allow for the usage of the spring platform transaction manager in more scenarios
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-109?page=com.atlassian.jira.plug... ]
Steven Hawkins updated TEIIDSB-109:
-----------------------------------
Fix Version/s: 1.4.0
(was: 1.3.0)
> Allow for the usage of the spring platform transaction manager in more scenarios
> --------------------------------------------------------------------------------
>
> Key: TEIIDSB-109
> URL: https://issues.redhat.com/browse/TEIIDSB-109
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: core
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.4.0
>
>
> Unless an xa source is being used, we would ideally use the default platform transaction manager. This would map updates over multiple sources to best effort single phase commit. This will likely be the majority of our usage.
> The problem is that the platform transaction manager doesn't directly support a suspend/resume concept that is needed for our remote pg/jdbc processing logic that allows the processing thread to detach from the current work when blocked and do something else.
> It would be worth investigating if this functionality could be added to spring boot or if there is is some other approach that was missed with the initial investigation.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (TEIIDSB-143) SAP Gateway returns 406
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-143?page=com.atlassian.jira.plug... ]
Steven Hawkins resolved TEIIDSB-143.
------------------------------------
Resolution: Done
1.3.0 has moved to Teiid 13 snapshot and will shortly move to final.
> SAP Gateway returns 406
> -----------------------
>
> Key: TEIIDSB-143
> URL: https://issues.redhat.com/browse/TEIIDSB-143
> Project: Teiid Spring Boot
> Issue Type: Bug
> Components: datasource
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.3.0
>
> Original Estimate: 2 hours, 30 minutes
> Time Spent: 2 hours
> Remaining Estimate: 30 minutes
>
> After the header fix TEIIDSB-134 for some reason SAP responds with a 406 - it doesn't like having multiple accept header values at least when used through our spring boot rest support. Just sending application/xml works as expected.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years