[JBoss JIRA] (TEIID-4271) Consume Teiid OData from Salesforce
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4271?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4271:
---------------------------------------
Since this will affect all vdbs, would this be better as an "exclude-system-schema" property that would prevent sys, sysadmin, and the pg schema from showing up?
> Consume Teiid OData from Salesforce
> -----------------------------------
>
> Key: TEIID-4271
> URL: https://issues.jboss.org/browse/TEIID-4271
> Project: Teiid
> Issue Type: Feature Request
> Components: OData
> Reporter: Steven Hawkins
> Assignee: Johnathon Lee
> Fix For: 8.12.x
>
>
> At least when consuming OData 2, Salesforce does not support Collection(Int16|Int32|String). Since we expose our system metadata, including pg metadata, in the OData metadata, this means that Salesforce won't consume. If the pg metadata is disabled and there are not primitive array columns in the rest of the metadata, then Salesforce can consume.
> Since this is working around a Salesforce issue, we shouldn't introduce too intrusive of a mechanism for this. And in general we may need to use extension metadata to allow users to exclude objects in a more granular way from being exposed in OData metadata.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-4271) Consume Teiid OData from Salesforce
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4271?page=com.atlassian.jira.plugin... ]
Ramesh Reddy reassigned TEIID-4271:
-----------------------------------
Assignee: Johnathon Lee (was: Ramesh Reddy)
> Consume Teiid OData from Salesforce
> -----------------------------------
>
> Key: TEIID-4271
> URL: https://issues.jboss.org/browse/TEIID-4271
> Project: Teiid
> Issue Type: Feature Request
> Components: OData
> Reporter: Steven Hawkins
> Assignee: Johnathon Lee
> Fix For: 8.12.x
>
>
> At least when consuming OData 2, Salesforce does not support Collection(Int16|Int32|String). Since we expose our system metadata, including pg metadata, in the OData metadata, this means that Salesforce won't consume. If the pg metadata is disabled and there are not primitive array columns in the rest of the metadata, then Salesforce can consume.
> Since this is working around a Salesforce issue, we shouldn't introduce too intrusive of a mechanism for this. And in general we may need to use extension metadata to allow users to exclude objects in a more granular way from being exposed in OData metadata.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-4271) Consume Teiid OData from Salesforce
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4271?page=com.atlassian.jira.plugin... ]
Ramesh Reddy updated TEIID-4271:
--------------------------------
Fix Version/s: 8.12.x
(was: 9.x)
> Consume Teiid OData from Salesforce
> -----------------------------------
>
> Key: TEIID-4271
> URL: https://issues.jboss.org/browse/TEIID-4271
> Project: Teiid
> Issue Type: Feature Request
> Components: OData
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 8.12.x
>
>
> At least when consuming OData 2, Salesforce does not support Collection(Int16|Int32|String). Since we expose our system metadata, including pg metadata, in the OData metadata, this means that Salesforce won't consume. If the pg metadata is disabled and there are not primitive array columns in the rest of the metadata, then Salesforce can consume.
> Since this is working around a Salesforce issue, we shouldn't introduce too intrusive of a mechanism for this. And in general we may need to use extension metadata to allow users to exclude objects in a more granular way from being exposed in OData metadata.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-4271) Consume Teiid OData from Salesforce
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4271?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4271:
-------------------------------------
The solution being provided is to exclude any schema from the $metadata document based on a configuration setting in the web.xml of teiid-odata.war file. The schema exclusion list can be made per VDB. To enable this, open the teiid-odata.war file using zip utility and un-comment following property
{code}
<context-param>
<param-name>exclude-schemas-from-metadata</param-name>
<param-value>vdb.1.SYS,vdb.1.SYSADMIN</param-value>
</context-param>
{code}
Add correct schema names (comma separated and fully qualified names vdbname.version.schemaname), update war file. The undeploy the old and redeploy the new teiid-odata.war file.
The commit to pull into the build can be found here https://github.com/rareddy/teiid/commit/8e6d7d05a6136aaf6eed241ed2452798a...
> Consume Teiid OData from Salesforce
> -----------------------------------
>
> Key: TEIID-4271
> URL: https://issues.jboss.org/browse/TEIID-4271
> Project: Teiid
> Issue Type: Feature Request
> Components: OData
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.x
>
>
> At least when consuming OData 2, Salesforce does not support Collection(Int16|Int32|String). Since we expose our system metadata, including pg metadata, in the OData metadata, this means that Salesforce won't consume. If the pg metadata is disabled and there are not primitive array columns in the rest of the metadata, then Salesforce can consume.
> Since this is working around a Salesforce issue, we shouldn't introduce too intrusive of a mechanism for this. And in general we may need to use extension metadata to allow users to exclude objects in a more granular way from being exposed in OData metadata.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-4227) Impala Translator - Support Multiple Count Distinct with Group By
by John Cockson (JIRA)
[ https://issues.jboss.org/browse/TEIID-4227?page=com.atlassian.jira.plugin... ]
John Cockson commented on TEIID-4227:
-------------------------------------
[~shawkins] If you are able to implement the suggestion [~walla2sl] puts forth that would be awesome. If a single count distinct is used the function would be pushed down (vast majority of requests I deal with) if more that one count distinct is required Teiid would remove the group by and handle processing it internally. I would also be behind generating a separate query for each count distinct and joining in Teiid. Either way this would be a awesome improvement. Thank you both for getting this started!
> Impala Translator - Support Multiple Count Distinct with Group By
> -----------------------------------------------------------------
>
> Key: TEIID-4227
> URL: https://issues.jboss.org/browse/TEIID-4227
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.13.3
> Reporter: Scott Wallace
> Assignee: Steven Hawkins
> Fix For: 9.1
>
>
> This is somewhat related to TEIID-3743, which added support of multiple count distinct metrics. We have a requirement to allow multiple count distinct metrics grouped by common attributes.
> Currently, we get a failure as follows:
> {noformat}org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidProcessingException: TEIID30504 vw_impression_click_transaction_process_date_detail: 500051 TEIID11008:TEIID11004 Error executing statement(s): [Prepared Values: [] SQL: SELECT g_0.process_date_key, SUM(g_0.num_clicks), SUM(g_0.num_impressions), COUNT(DISTINCT g_0.orderid), COUNT(DISTINCT g_0.`hash_tid`), SUM(g_0.sales) FROM detail.vw_impression_click_transaction_process_date_detail g_0 WHERE g_0.process_date_key >= '2016-05-22' AND g_0.process_date_key < '2016-05-23' AND g_0.advertiser_key = 12345 GROUP BY g_0.process_date_key]
> {noformat}
> To allow this request to succeed, we could have translator write that Impala query as:
> {noformat}
> select nvl(v1.c1,v2.c1), nvl(v1.c2,v1.c2), nvl(v1.c3,v1.c3), v1.c4, v2.c5, nvl(v1.c6,v1.c6) from (
> SELECT g_0.process_date_key as c1,
> SUM(g_0.num_clicks) as c2,
> SUM(g_0.num_impressions) as c3,
> COUNT(DISTINCT g_0.orderid) as c4,
> SUM(g_0.sales) as c6
> FROM detail.vw_impression_click_transaction_process_date_detail g_0
> WHERE g_0.process_date_key >= '2016-05-22'
> AND g_0.process_date_key < '2016-05-23'
> AND g_0.advertiser_key = 12345
> GROUP BY g_0.process_date_key
> ) v1 inner join
> (
> SELECT g_0.process_date_key as c1,
> SUM(g_0.num_clicks) as c2,
> SUM(g_0.num_impressions) as c3,
> COUNT(DISTINCT g_0.`hash_tid`) as c5,
> SUM(g_0.sales) as c6
> FROM detail.vw_impression_click_transaction_process_date_detail g_0
> WHERE g_0.process_date_key >= '2016-05-22'
> AND g_0.process_date_key < '2016-05-23'
> AND g_0.advertiser_key = 12345
> GROUP BY g_0.process_date_key
> ) v2 on v1.c1 = v2.c1;
> {noformat}
> FYI [~shawkins]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-4277) Infinispan Cache DSL translator: materialization incomplete results during view's loading
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-4277?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-4277:
------------------------------------------------
Andrej Smigala <asmigala(a)redhat.com> changed the Status of [bug 1347582|https://bugzilla.redhat.com/show_bug.cgi?id=1347582] from ON_QA to VERIFIED
> Infinispan Cache DSL translator: materialization incomplete results during view's loading
> -----------------------------------------------------------------------------------------
>
> Key: TEIID-4277
> URL: https://issues.jboss.org/browse/TEIID-4277
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 9.x, 8.12.5
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Blocker
> Fix For: 9.1, 8.12.5
>
> Attachments: test.log
>
>
> When user tries to query materialized view in a moment when its loading is in progress, incomplete results are returned. The status of the view in corresponding status table is LOADING.
> Vdb used:
> {code:xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <vdb name="jdg-remote" version="1">
> <description>For testing of materialized views</description>
> <property name ="lib" value ="org.jboss.qe.jdg.pojos"></property>
> <model name="jdgSource">
> <source name="jdgSource" translator-name="jdg-override"
> connection-jndi-name="java:/jdg-mat-ds" />
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE CustomerReport(
> customerId integer not null OPTIONS (SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> totalAmount integer,
> CONSTRAINT PK_ID PRIMARY KEY(customerId)
> ) OPTIONS (NAMEINSOURCE '${db.table.prefix}jdg-mat', UPDATABLE TRUE);]]>
> </metadata>
> </model>
> <model name="h2Source">
> <source name="h2Source" translator-name="h2" connection-jndi-name="java:/h2-ds" />
> </model>
> <model name="View" type="VIRTUAL">
> <metadata type="DDL"><![CDATA[
> CREATE VIEW external_long_ttl (
> customerId integer NOT NULL,
> totalAmount integer
> ) OPTIONS (MATERIALIZED 'TRUE',
> MATERIALIZED_TABLE 'jdgSource.CustomerReport',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO jdgSource.CustomerReport(customerId,totalAmount) SELECT c.id AS customerId, CONVERT(SUM(o.amount),integer) AS totalAmount FROM DV_MATVIEWS_CUSTOMERS c INNER JOIN DV_MATVIEWS_ORDERS o ON c.id = o.customer_id GROUP BY c.id;',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'h2Source.DB.PUBLIC.DV_MATVIEWS_STATUSTABLE',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute jdgSource.native(''truncate cache'');',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'execute jdgSource.native(''swap cache names'');',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
> "teiid_rel:MATVIEW_TTL" 10000
> )
> AS SELECT c.id AS customerId, CONVERT(SUM(o.amount),integer) AS totalAmount FROM DV_MATVIEWS_CUSTOMERS c INNER JOIN DV_MATVIEWS_ORDERS o ON c.id = o.customer_id GROUP BY c.id;
> ]]>
> </metadata>
> </model>
> <translator name="jdg-override" type="infinispan-cache-dsl">
> <property name="SupportsDirectQueryProcedure" value="true" />
> </translator>
> </vdb>
> {code}
> Resource-adapter used:
> {code:xml}
> <resource-adapter id="jdg-mat">
> <module slot="main" id="org.jboss.teiid.resource-adapter.infinispan.dsl"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.infinispan.dsl.InfinispanManagedConnectionFactory" jndi-name="java:/jdg-mat-ds" enabled="true" pool-name="jdg-mat">
> <config-property name="ProtobufDefinitionFile">
> /org/jboss/qe/jdg/remote/protobuf/CustomerReport.proto
> </config-property>
> <config-property name="AliasCacheName">
> ${db.table.prefix}jdg-alias
> </config-property>
> <config-property name="MessageMarshallers">
> org.jboss.qe.jdg.pojo.CustomerReport:org.jboss.qe.jdg.remote.marshaller.CustomerReportMarshaller
> </config-property>
> <config-property name="RemoteServerList">
> [remote-server-address]:11322
> </config-property>
> <config-property name="StagingCacheName">
> ${db.table.prefix}jdg-mat-stage
> </config-property>
> <config-property name="CacheTypeMap">
> ${db.table.prefix}jdg-mat:org.jboss.qe.jdg.pojo.CustomerReport;customerId
> </config-property>
> <config-property name="MessageDescriptor">
> org.jboss.qe.jdg.remote.protobuf.CustomerReport
> </config-property>
> <config-property name="Module">
> org.jboss.qe.jdg.pojos
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> Datasource used (tables that the view queries):
> {code:xml}
> <xa-datasource jndi-name="java:/h2-ds" pool-name="h2-xa-ds" enabled="true">
> <xa-datasource-property name="URL">
> jdbc:h2:tcp://localhost:9292/db;TRACE_LEVEL_FILE=4
> </xa-datasource-property>
> <driver>h2</driver>
> <security>
> <user-name>sa</user-name>
> </security>
> </xa-datasource>
> {code}
> Query invoked to check current contents of materialized view:
> {code:sql}
> SELECT COUNT(*) FROM external_long_ttl;
> {code}
> And expected result is the same as count of DV_MATVIEWS_CUSTOMERS table from source h2Source. In my case 10.
> But when I print out the result I get all numbers from 0 to 10 . See attached log output (search for 'check-count:')
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-2578) add/remove schema elements
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2578?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2578:
-------------------------------------
I will be modifying the schema creation as
{code}
CREATE (VIRTUAL|FOREIGN) SCHEMA <name> [SERVER <server-name> (<COMMA> <server-name>)*] OPTIONS (....)
{code}
This will establish two things.
1. Defines the SCHEMA aka model is PHYSICAL or VIRTUAL.
2. Associates the Data Source name(s) with a SCHEMA. This being plural, will also handle the Multi Source models.
> add/remove schema elements
> --------------------------
>
> Key: TEIID-2578
> URL: https://issues.jboss.org/browse/TEIID-2578
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 9.1
>
>
> Schemas are currently static after load. Modifications can only happen with restarts or new versions. We should allow add/drop at runtime.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (TEIID-4271) Consume Teiid OData from Salesforce
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4271?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4271:
-------------------------------------
[~shawkins] I believe in OData v2, there is limited support for collection types, OData V3 added some of it. OData V4 lots of stuff uses collection types, so an implementation not supporting collections is just not usable IMO.
So, considering this applies to only OData V2, we can target this to JDV 6.3 scoped build. May be we should just reject here and work with BZ.
> Consume Teiid OData from Salesforce
> -----------------------------------
>
> Key: TEIID-4271
> URL: https://issues.jboss.org/browse/TEIID-4271
> Project: Teiid
> Issue Type: Feature Request
> Components: OData
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.x
>
>
> At least when consuming OData 2, Salesforce does not support Collection(Int16|Int32|String). Since we expose our system metadata, including pg metadata, in the OData metadata, this means that Salesforce won't consume. If the pg metadata is disabled and there are not primitive array columns in the rest of the metadata, then Salesforce can consume.
> Since this is working around a Salesforce issue, we shouldn't introduce too intrusive of a mechanism for this. And in general we may need to use extension metadata to allow users to exclude objects in a more granular way from being exposed in OData metadata.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months