[teiid-issues] [JBoss JIRA] (TEIID-4284) Implement Salesforce Bulk API for SELECTS to Salesforce Connector

[sameer P (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13306083#comment-13306083 ] 

sameer P edited comment on TEIID-4284 at 10/13/16 6:28 AM:
-----------------------------------------------------------

[~shawkins], I checked the batch status after the second execution of the same query and it had mainly two batches: one with the *Not Processed* state (original batch) and the other with *COMPLETED* status (subsequent batch). And from salesforce, the status *Not Processed* means:

{code:java}
The batch won't be processed. This state is assigned when a job is aborted while the batch is queued. For bulk queries, if the job has PK chunking enabled, this state is assigned to the original batch that contains the query when the subsequent batches are created. After the original batch is changed to this state, you can monitor the subsequent batches and retrieve each batch's results when it's completed.
{code}

So, may be instead of waiting for just *COMPLETED* status, we should also check the status of subsequent batches in the Job? 


was (Author: sameerp):
[~shawkins], I checked the bulk status after the second execution of the same query and it had mainly two bulks : one with the *Not Processed* state (original batch) and the other with *COMPLETED* status (subsequent batch). And from salesforce, the status *Not Processed* means:

{code:java}
The batch won't be processed. This state is assigned when a job is aborted while the batch is queued. For bulk queries, if the job has PK chunking enabled, this state is assigned to the original batch that contains the query when the subsequent batches are created. After the original batch is changed to this state, you can monitor the subsequent batches and retrieve each batch's results when it's completed.
{code}

So, may be instead of waiting for just *COMPLETED* status, we should also check the status of subsequent bulks in the Job? 

> Implement Salesforce Bulk API for SELECTS to Salesforce Connector
> -----------------------------------------------------------------
>
>                 Key: TEIID-4284
>                 URL: https://issues.jboss.org/browse/TEIID-4284
>             Project: Teiid
>          Issue Type: Feature Request
>          Components: Salesforce Connector
>    Affects Versions: 8.13.5
>         Environment: With Salesforce datasource
>            Reporter: sameer P
>            Assignee: Steven Hawkins
>             Fix For: 9.1
>
>
> There is some huge data (many GBs) in the Salesforce which has around 1.5 million rows and doing some simple select * on it fails with QUERY_TIMEOUT.  
> The salesforce guys suggested to try Bulk API for select with PK chunking as stated in https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/asynch_api_code_curl_walkthrough_pk_chunking.htm . 



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)