October 2016 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin... ] Steven Hawkins reopened TEIID-4284: ----------------------------------- Sameer, I'm not completely able to reproduce the behavior that you are seeing, but I do see that the current implementation does not do pk chunking. Rather it's just the bulk implementation. I have this mostly working, but not in time for the release so I'm pulling from 9.1 for further work in 9.2. > Implement Salesforce Bulk API for SELECTS to Salesforce Connector > ----------------------------------------------------------------- > > Key: TEIID-4284 > URL: https://issues.jboss.org/browse/TEIID-4284 > Project: Teiid > Issue Type: Feature Request > Components: Salesforce Connector > Affects Versions: 8.13.5 > Environment: With Salesforce datasource > Reporter: sameer P > Assignee: Steven Hawkins > Fix For: 9.1 > > > There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT. > The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asy... . -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4455) Impala Translator - Change planning step for from_unixtime() pushdown

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-4455?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-4455: --------------------------------------- Yes, this is a change in the engine logic starting in the QueryRewriter. The rewriter is replacing from_unixtime with the timestampadd equivalent - that needs moved to be a much later operation probably in LanguageBridgeFactory based upon the actual support. Also the logic for function support needs to be updated so that pushdown support for from_unixtime checks both for from_unixtime and timestampadd. Finally from_unix time needs to be added as a SourceSystemFunction and added to the getSupportedFunctions list for hive/impala. > Impala Translator - Change planning step for from_unixtime() pushdown > --------------------------------------------------------------------- > > Key: TEIID-4455 > URL: https://issues.jboss.org/browse/TEIID-4455 > Project: Teiid > Issue Type: Feature Request > Components: JDBC Connector > Reporter: Don Krapohl > Assignee: Kylin Soong > Priority: Minor > Labels: Impala_Translator > Fix For: 9.2 > > > Impala and Teiid both have from_unixtime() functions and per the forum reference the mapping to timestampadd() appears to come before the test for pushdown support of the impala native function. > As a query request we require the impala from_unixtime() function be executed instead of the Teiid-native version. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4511) st_Intersection not implemented

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-4511?page=com.atlassian.jira.plugin... ] Steven Hawkins resolved TEIID-4511. ----------------------------------- Resolution: Done Merged Andrej's commit and updated the translator pushdowns / docs as well. > st_Intersection not implemented > ------------------------------- > > Key: TEIID-4511 > URL: https://issues.jboss.org/browse/TEIID-4511 > Project: Teiid > Issue Type: Bug > Components: Query Engine > Affects Versions: 9.1 > Reporter: Andrej Šmigala > Assignee: Steven Hawkins > Fix For: 9.1 > > > The st_Intersection function was supposed to be implemented as part of TEIID-4393, but was omitted -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4513) Create a dummy pg_prepared_xacts table

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-4513?page=com.atlassian.jira.plugin... ] Steven Hawkins resolved TEIID-4513. ----------------------------------- Resolution: Done Added a dummy table that will produce no rows. > Create a dummy pg_prepared_xacts table > -------------------------------------- > > Key: TEIID-4513 > URL: https://issues.jboss.org/browse/TEIID-4513 > Project: Teiid > Issue Type: Sub-task > Components: ODBC > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Fix For: 9.2 > > > We should provide a dummy table to prevent errors from recovery scans. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

by sameer P (JIRA)

[ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin... ] sameer P edited comment on TEIID-4284 at 10/13/16 12:12 PM: ------------------------------------------------------------ Yes exactly Steven, and one more interesting thing is that, as I went through the Salesforce web UI which shows the number of batches that will be created for a submitted query, there will always be a single batch that will be created on first query (irrespective of how big the table is). And from the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? was (Author: sameerp): Yes exactly Steven, and one more interesting thing is that, there will always be a single batch that will be created on first query (irrespective of how big the table is) and from the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? > Implement Salesforce Bulk API for SELECTS to Salesforce Connector > ----------------------------------------------------------------- > > Key: TEIID-4284 > URL: https://issues.jboss.org/browse/TEIID-4284 > Project: Teiid > Issue Type: Feature Request > Components: Salesforce Connector > Affects Versions: 8.13.5 > Environment: With Salesforce datasource > Reporter: sameer P > Assignee: Steven Hawkins > Fix For: 9.1 > > > There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT. > The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asy... . -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4505) Clarify JDG doc's

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-4505?page=com.atlassian.jira.plugin... ] Steven Hawkins reassigned TEIID-4505: ------------------------------------- Fix Version/s: 9.2 Assignee: Van Halbert (was: Steven Hawkins) > Clarify JDG doc's > ----------------- > > Key: TEIID-4505 > URL: https://issues.jboss.org/browse/TEIID-4505 > Project: Teiid > Issue Type: Task > Components: Misc. Connectors > Affects Versions: 9.x > Reporter: Van Halbert > Assignee: Van Halbert > Priority: Minor > Fix For: 9.2 > > > In the JDG related doc's, clarify the following: > JDG Hot Rod connector > - when defining the cacheTypeMap, the pfKeyField name should correspond to a getter/setter method in the pojo. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

by sameer P (JIRA)

[ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin... ] sameer P edited comment on TEIID-4284 at 10/13/16 11:54 AM: ------------------------------------------------------------ Yes exactly Steven, and one more interesting thing is that, there will always be a single batch that will be created on first query (irrespective of how big the table is) and from the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? was (Author: sameerp): Yes Exactly Steven, and one more interesting thing is that, there will always be a single batch that will be created on first query (irrespective of how big the table is) and from the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? > Implement Salesforce Bulk API for SELECTS to Salesforce Connector > ----------------------------------------------------------------- > > Key: TEIID-4284 > URL: https://issues.jboss.org/browse/TEIID-4284 > Project: Teiid > Issue Type: Feature Request > Components: Salesforce Connector > Affects Versions: 8.13.5 > Environment: With Salesforce datasource > Reporter: sameer P > Assignee: Steven Hawkins > Fix For: 9.1 > > > There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT. > The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asy... . -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

by sameer P (JIRA)

[ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin... ] sameer P edited comment on TEIID-4284 at 10/13/16 11:51 AM: ------------------------------------------------------------ Yes Exactly Steven, and one more interesting thing is that, there will always be a single batch that will be created on first query (irrespective of how big the table is) and from the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? was (Author: sameerp): Yes Exactly Steven, and one more interesting thing is that, there will always be a single batch that will be created on first query (irrespective of how big the table is) and on the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? > Implement Salesforce Bulk API for SELECTS to Salesforce Connector > ----------------------------------------------------------------- > > Key: TEIID-4284 > URL: https://issues.jboss.org/browse/TEIID-4284 > Project: Teiid > Issue Type: Feature Request > Components: Salesforce Connector > Affects Versions: 8.13.5 > Environment: With Salesforce datasource > Reporter: sameer P > Assignee: Steven Hawkins > Fix For: 9.1 > > > There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT. > The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asy... . -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

by sameer P (JIRA)

[ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin... ] sameer P commented on TEIID-4284: --------------------------------- Yes Exactly Steven, and one more interesting thing is that, there will always be a single batch that will be created on first query (irrespective of how big the table is) and on the second execution of the same query on wards, the query will be distributed into batches. Is this the expected behavior? > Implement Salesforce Bulk API for SELECTS to Salesforce Connector > ----------------------------------------------------------------- > > Key: TEIID-4284 > URL: https://issues.jboss.org/browse/TEIID-4284 > Project: Teiid > Issue Type: Feature Request > Components: Salesforce Connector > Affects Versions: 8.13.5 > Environment: With Salesforce datasource > Reporter: sameer P > Assignee: Steven Hawkins > Fix For: 9.1 > > > There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT. > The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asy... . -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-4284: --------------------------------------- Yes, it looks like we should be able to get the batch info list in the Not Processed. It seems to also imply that the batches then may individually not be complete, so the logic for getting each batch may need adjusted as well. > Implement Salesforce Bulk API for SELECTS to Salesforce Connector > ----------------------------------------------------------------- > > Key: TEIID-4284 > URL: https://issues.jboss.org/browse/TEIID-4284 > Project: Teiid > Issue Type: Feature Request > Components: Salesforce Connector > Affects Versions: 8.13.5 > Environment: With Salesforce datasource > Reporter: sameer P > Assignee: Steven Hawkins > Fix For: 9.1 > > > There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT. > The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asy... . -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 2 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues October 2016