[
https://issues.jboss.org/browse/TEIID-2316?page=com.atlassian.jira.plugin...
]
Steven Hawkins resolved TEIID-2316.
-----------------------------------
Resolution: Done
Removed the connector batch size setting and added a release note.
We will now push twice the working batch size on the execution context. The working batch
size is determined from the processor batch size and the estimated data size. We'll
adjust the working batch size up or down from 1/8 to 8x the processor batch size. With
the default processor batch size is 256, which means that the connector will be asked for
batch sizes between 64 and 4096 rows.
For most scenarios this should be sufficient to prevent issues with allocating too much
space based upon a single connector batch size setting.
The google connector was modified so that batch size refers to the max batch size.
There may need to be some other provisions for sources, such as oracle that perfer
extremely small fetch sizes.
Also updated the access node logic not to block if partial batches are delivered to it
(which can also happen in continuous mode). This allows translators to effectively ignore
the batch size and deliver whatever number of rows they feel is appropriate.
Give translators a better batch size
------------------------------------
Key: TEIID-2316
URL:
https://issues.jboss.org/browse/TEIID-2316
Project: Teiid
Issue Type: Enhancement
Components: Query Engine
Affects Versions: 8.3
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 8.3
7.5 and later releases refined logic in the engine to shape batch sizes based upon the
data, however we still only use the default connector batch size when requesting data.
This value currently defaults to 512, however that can be much too large for many
scenarios given how the underlying sources handling buffering. At the least we should
provide the engine adjusted batch size. It would be good as well to let the translator
specify a scaling factor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:
http://www.atlassian.com/software/jira