[teiid-issues] [JBoss JIRA] (TEIID-3272) JDBC Translator | Segregation of batch execution and commit mode

[Shiveeta Mattoo (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029865#comment-13029865 ] 

Shiveeta Mattoo edited comment on TEIID-3272 at 1/5/15 8:46 AM:
----------------------------------------------------------------

The whole intent of the configurable batch commit size is to ensure that although we are capable of using the performance benefit offered by batching the SQL statements, we are able to control the atomicity by configuring the "batch count to commit".

Few scenarios around expected behaviour :

1. Batch size = 100, Batch count to commit = 2, Input record count = 500
If Batch # 3 has any error, then Batch#1,2 will be committed and whole of Batch # 3 will be rolled back, and no further batches processed.

2. Batch size = 100, Batch count to commit = 5, Input record count = 500
If any failure is last batch # 5, none of the records are committed, as all 500 records are logically one atomic transation


3.Batch size = 100, Batch count to commit = 2, Input record count = 500
 Provide user an additional -- continueOnError option. If this option is enabled, we will do a Best Effort processing , So only error records are skipped in this case, and everything else is committed.
The actual implementation of this feature would depend on what features the underlying database / source provides for error handling.



was (Author: shiveeta.mattoo):
The whole intent of the configurable batch commit size is to ensure that although we are capable of using the performance benefit offered by batching the SQL statements, we are able to control the atomicity by configuring the "batch count to commit".

Few scenarios around expected behaviour :

1. Batch size = 100, Batch count to commit = 2, Input record count = 500
If Batch # 3 has any error, then Batch#1,2 will be committed and whole of Batch # 3 will be rolled back, and no further batches processed.

2. Batch size = 100, Batch count to commit = 5, Input record count = 500
If any failure is last batch # 5, none of the records are committed, as all 500 records are logically one atomic transation


3.Batch size = 100, Batch count to commit = 2, Input record count = 500
 Provide user an additional -- continueOnError option. If this option is enabled we will process the erroneous batch till the erroneous record is reached :
If Batch#3 has any error,  a Savepoint is set at end of Batch#2 and further records pf batch# 3 are executed and committed one by one till the erroneous record is reached and then processing continues after the failed batch.

So only error records can be skipped in this case


> JDBC Translator | Segregation of batch execution and commit mode 
> -----------------------------------------------------------------
>
>                 Key: TEIID-3272
>                 URL: https://issues.jboss.org/browse/TEIID-3272
>             Project: Teiid
>          Issue Type: Enhancement
>          Components: JDBC Connector
>            Reporter: Shiveeta Mattoo
>            Assignee: Steven Hawkins
>              Labels: Batch-Commit, JDBC-Transator
>         Attachments: batch1.png
>
>
> In JDBC Translator, the batch execution and commit are coupled together. Hence, we after every batch execution, a commit is executed. 
> This enhancement is for providing an ability to support the commit after a configurable 'x' no. of batches has completed, instead of an automatic commit after every batch.
> The no of batches after which a commit is desired should be a configurable option.
>  



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)