Yoann Rodière (
https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%...
) *created* an issue
Hibernate Search (
https://hibernate.atlassian.net/browse/HSEARCH?atlOrigin=eyJpIjoiY2RjZjQ3...
) / Improvement (
https://hibernate.atlassian.net/browse/HSEARCH-3871?atlOrigin=eyJpIjoiY2R...
) HSEARCH-3871 (
https://hibernate.atlassian.net/browse/HSEARCH-3871?atlOrigin=eyJpIjoiY2R...
) Always enable delayed commits in the Lucene backend (
https://hibernate.atlassian.net/browse/HSEARCH-3871?atlOrigin=eyJpIjoiY2R...
)
Issue Type: Improvement Assignee: Yoann Rodière (
https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%...
) Components: backend-lucene Created: 25/Mar/2020 09:50 AM Fix Versions:
6.0.0.Beta-backlog-high-priority Priority: Major Reporter: Yoann Rodière (
https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%...
)
Let's start by saying that delayed commits are nothing to be afraid of. When they are
enabled, write operations that require an immediate commit (e.g. automatic indexing by
default) are still committed immediately. The delayed commits are only effective when some
write operations do *not* perform an immediate commit. In that case, the user already
decided to take the chance of losing data in the event of a crash, so waiting 1 second
before the commit will not change much to the data safety.
It does change much when it comes to performance, though. Instead of committing e.g. every
100 operations, we commit e.g. every second: maybe no additional operation will be
performed by then, in which case which case performance is the same, but maybe thousands
of write operations will have be performed by then, in which case we just reduced commits
by a factor 10... and probably increased throughput by the same factor.
With that in mind... *not* enabling delayed commits is becoming a problem. Early numbers
from the performance tests on HSEARCH-3822 (
https://hibernate.atlassian.net/browse/HSEARCH-3822 ) In Progress show that any
improvement to the performance when delayed commits are enabled will probably lead to
massive performance regressions when delayed commits are *not* enabled. We're talking
about doubling the throughput when delayed commit are enabled, while dividing it by a
factor 10 when delayed commits are disabled.
The reason for the regressions is that we have to commit after each batch of writes when
delayed commits are disabled, and the improvements are about performing multiple, but
smaller batche of writes in parallel. So when delayed commits are disabled, the
"improvements" actually increase the amount of commits. When delayed commits are
enabled, they do not.
(
https://hibernate.atlassian.net/browse/HSEARCH-3871#add-comment?atlOrigin...
) Add Comment (
https://hibernate.atlassian.net/browse/HSEARCH-3871#add-comment?atlOrigin...
)
Get Jira notifications on your phone! Download the Jira Cloud app for Android (
https://play.google.com/store/apps/details?id=com.atlassian.android.jira....
) or iOS (
https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=Em...
) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100122- sha1:092c6a7 )