[hibernate-commits] [hibernate/hibernate-search] 1b16f5: HSEARCH-3872 Remove the minimum number of works in...
Yoann Rodière
noreply at github.com
Mon Mar 30 07:15:39 EDT 2020
Branch: refs/heads/master
Home: https://github.com/hibernate/hibernate-search
Commit: 1b16f546e5d5bccbb772171815074eb2169ae8ce
https://github.com/hibernate/hibernate-search/commit/1b16f546e5d5bccbb772171815074eb2169ae8ce
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkBulker.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/work/ElasticsearchIndexingIT.java
Log Message:
-----------
HSEARCH-3872 Remove the minimum number of works in an Elasticsearch bulk
Always bulk what can be bulked.
The goal is to make work execution simpler.
Commit: 6f109d1e229fded50ff64c124c50b5b6781e9171
https://github.com/hibernate/hibernate-search/commit/6f109d1e229fded50ff64c124c50b5b6781e9171
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkExecutionContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchImmutableWorkExecutionContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchRefreshableWorkExecutionContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkExecutionContextImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSequenceBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/query/impl/ElasticsearchSearchQueryImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/BulkWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ClearScrollWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CloseIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CountWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CreateIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteByQueryWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DropIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ElasticsearchWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ExplainWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/FlushWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/GetIndexMetadataWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/MergeSegmentsWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/OpenIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexAliasesWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexMappingWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexSettingsWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/RefreshWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ScrollWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/SearchWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/WaitForIndexStatusWorkBuilder.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableElasticsearchWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSimpleElasticsearchWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentElasticsearchWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ClearScrollWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CloseIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CountWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CreateIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteByQueryWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DropIndexWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchForwardingWorkExecutionContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkAggregator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkExecutionContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ExplainWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/FlushWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ForceMergeWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/GetIndexMetadataWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/NonBulkableElasticsearchWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/OpenIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexAliasesWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexSettingsWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexTypeMappingWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/RefreshWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ScrollWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SearchWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/WaitForIndexStatusWork.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/mapping/ElasticsearchFieldAttributesIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/mapping/ElasticsearchFieldTypesIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/mapping/ElasticsearchTypeNameMappingSchemaIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/work/ElasticsearchIndexingIT.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch5TestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch60TestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch63TestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch64TestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch67TestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch7TestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/ElasticsearchTestDialect.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/rule/TestElasticsearchClient.java
Log Message:
-----------
HSEARCH-3872 Remove the ability to execute bulkable works outside of a bulk
Just execute them in a bulk, always. It's simpler, and more importantly
it allows us to get rid of the end-of-sequence refresh.
That end-of-sequence refresh must disappear if we want to have multiple
work queues and threads per index, sharded by document ID.
In that case, works from a single work plan may be executed in different
threads, thus we can no longer expect the refresh to be executed
"just after the sequence".
For now we'll execute refreshes as part of the bulk works, which may
incur a significant performance cost, as we may execute multiple
refreshes per index for each work plan.
We may want to look into this in the next commits, but since forced
refreshes already perform very badly, this is not a priority.
Commit: 89e702f27a9a787d2a06a9778334447f9095c621
https://github.com/hibernate/hibernate-search/commit/89e702f27a9a787d2a06a9778334447f9095c621
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/Elasticsearch67WorkBuilderFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/Elasticsearch7WorkBuilderFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/ElasticsearchWorkBuilderFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexer.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java
Log Message:
-----------
HSEARCH-3872 Do not URL-encode document identifiers for Elasticsearch indexing works
Now that works are always bulked, we do not need URL-encoding anymore.
Commit: 6f2f84dc69ac76191e69541dcc9e482fd1fecedc
https://github.com/hibernate/hibernate-search/commit/6f2f84dc69ac76191e69541dcc9e482fd1fecedc
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkBulker.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSequenceBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/Elasticsearch7WorkBuilderFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/ElasticsearchWorkBuilderFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/BulkWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ClearScrollWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CloseIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CountWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CreateIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteByQueryWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DropIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ExplainWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/FlushWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/GetIndexMetadataWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/IndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/MergeSegmentsWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/OpenIndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexAliasesWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexMappingWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexSettingsWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/RefreshWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ScrollWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/SearchWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/WaitForIndexStatusWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSet.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableElasticsearchWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentElasticsearchWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkableElasticsearchWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkableWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ClearScrollWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CloseIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CountWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CreateIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteByQueryWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DropIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkAggregator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ExplainWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/FlushWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ForceMergeWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/GetIndexMetadataWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/NonBulkableElasticsearchWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/NonBulkableWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/OpenIndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexAliasesWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexSettingsWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexTypeMappingWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/RefreshWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ScrollWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SearchWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentElasticsearchWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/WaitForIndexStatusWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/result/impl/BulkResultItemExtractor.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWorkTest.java
Log Message:
-----------
HSEARCH-3872 Remove an unnecessary "Elasticsearch" component in the class name of Elasticsearch works
Commit: a062bef7e3de32bd86e5960a43046680c8007b7b
https://github.com/hibernate/hibernate-search/commit/a062bef7e3de32bd86e5960a43046680c8007b7b
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/IndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSet.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentWork.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java
Log Message:
-----------
HSEARCH-3872 Give a common return type to all works implementing SingleDocumentWriteWork
This will be useful when manipulating futures and single-document works
in the next commits.
Commit: 9cd4135b36bd04a0ff6d2d99c85e60a47cae25a0
https://github.com/hibernate/hibernate-search/commit/9cd4135b36bd04a0ff6d2d99c85e60a47cae25a0
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/logging/impl/Log.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java
Log Message:
-----------
HSEARCH-3872 Remove the requirement for works within a write plan to be skipped when a previous work failed
Because:
1. It's not necessary. The only works added to a write plan are add,
update, delete, and we expect to be able to execute these works even
if a previous add/update/delete failed.
2. It won't be possible anymore once we organize work execution in
multiple queues sharded by ID, where we will lose the information
"this work belongs to the same write plan as this other work".
Commit: fb00b50300c07832a23277e42ed1a36d4c1120c3
https://github.com/hibernate/hibernate-search/commit/fb00b50300c07832a23277e42ed1a36d4c1120c3
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSingleWorkSet.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecution.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSet.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java
A backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecutionTest.java
R backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java
Log Message:
-----------
HSEARCH-3872 Remove the concept of "work set" from Elasticsearch work processors
Only ever process works one by one.
This is necessary if we want to have multiple work queues and threads
per index, sharded by document ID. In that case, works from a single
work plan may be executed in different threads, thus we can no longer
pass "work sets" to the processors.
We need something to be notified that all works have been executed, and
to act accordingly.
Commit: 3d562d20ce1f19cecfe3be261accd0f10c010668
https://github.com/hibernate/hibernate-search/commit/3d562d20ce1f19cecfe3be261accd0f10c010668
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkBulker.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java
Log Message:
-----------
HSEARCH-3872 Add works to the sequence automatically in ElasticsearchWorkBulker
In the past this was not possible and we had to expose
addWorksToSequence() in order for clients to add the works explicitly,
but with the removals of the past few commits, we can simplify that.
Commit: 9a982b0e59c7a63a68fb73d95da18b3a080d841e
https://github.com/hibernate/hibernate-search/commit/9a982b0e59c7a63a68fb73d95da18b3a080d841e
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSequenceBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/result/impl/BulkResult.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/result/impl/BulkResultItemExtractor.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWorkTest.java
Log Message:
-----------
HSEARCH-3872 Add bulk extraction through a single method call in ElasticsearchWorkSequenceBuilder
In the past this was not possible and we had to expose
an intermediary object to add extractions
(ElasticsearchWorkSequenceBuilder.BulkResultExtractionStep),
but with the removals of the past few commits, we can simplify that.
Commit: aa1c39fe5fcd9ff650c02f87dd2e34c5d20dfdaa
https://github.com/hibernate/hibernate-search/commit/aa1c39fe5fcd9ff650c02f87dd2e34c5d20dfdaa
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/AbstractElasticsearchWorkOrchestrator.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSingleWorkSet.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSet.java
Log Message:
-----------
HSEARCH-3872 Remove the concept of "work set" in the Elasticsearch backend
Work sets no longer make sense, at least during work orchestration,
since we submit works one by one without any information about the
relationship between works.
Commit: 0ef199359633b090287bb3e78158c183bb36ba6f
https://github.com/hibernate/hibernate-search/commit/0ef199359633b090287bb3e78158c183bb36ba6f
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/impl/ElasticsearchClientFactoryImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/impl/ElasticsearchClientImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/spi/ElasticsearchClientFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchLinkImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/IndexManagerBackendContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/resources/impl/BackendThreads.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchClientFactoryImplIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchContentLengthIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/testsupport/util/ElasticsearchClientSpy.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/rule/TestElasticsearchClient.java
Log Message:
-----------
HSEARCH-3872 Use a single thread pool for all Elasticsearch indexes
It may reduce performance when there are many indexes,
but it brings the number of threads to a more reasonable amount in that
case, and it'll allow us to introduce sharding so as to perform better
when a single index is getting hit by many write requests.
Commit: 7659094b0b737a71a3c6ea64d8cbdd2d24ec952d
https://github.com/hibernate/hibernate-search/commit/7659094b0b737a71a3c6ea64d8cbdd2d24ec952d
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/writer/impl/IndexWriterDelegatorImpl.java
M engine/src/main/java/org/hibernate/search/engine/backend/orchestration/spi/BatchingExecutor.java
M engine/src/main/java/org/hibernate/search/engine/backend/orchestration/spi/SingletonTask.java
Log Message:
-----------
HSEARCH-3872 Do not block when waiting for work completion in SingletonTask/BatchingExecutor
So that the thread pool can be used for other tasks.
This is especially important for the Elasticsearch backend, where we
execute work asynchronously with a parallelism equal to the number of
connections, and have potentially much more connections available than
the number of threads in the pool.
Commit: 11a7019eabdd7f5617512fc838e1f327cf2fdc21
https://github.com/hibernate/hibernate-search/commit/11a7019eabdd7f5617512fc838e1f327cf2fdc21
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/ElasticsearchIndexManagerImpl.java
Log Message:
-----------
HSEARCH-3872 Execute mass indexing works in the serial orchestrator in the Elasticsearch backend
Because we need to preserve the order of these works relative to
indexing plan works, as much as possible.
This may decrease performance of mass indexing on Elasticsearch, but
we'll bring it back up by sharding work queues for indexing in the
next commits.
Commit: f150c2ee1c317b32f2535c4f4036be18a8816780
https://github.com/hibernate/hibernate-search/commit/f150c2ee1c317b32f2535c4f4036be18a8816780
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/ElasticsearchIndexManagerImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/IndexManagerBackendContext.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/AbstractElasticsearchWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkOrchestrator.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkOrchestrator.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSimpleWorkOrchestrator.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorImplementor.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/schema/management/impl/ElasticsearchIndexSchemaManager.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/schema/management/impl/ElasticsearchSchemaAccessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/query/impl/ElasticsearchSearchQueryBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/query/impl/ElasticsearchSearchQueryImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexer.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecution.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexWorkspace.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/WorkExecutionBackendContext.java
A backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessorTest.java
R backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java
R backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecutionTest.java
Log Message:
-----------
HSEARCH-3872 Do not batch Elasticsearch works when executing them in parallel
No need for a batching orchestrator, as we don't ever bulk these works
and we always expect the user to wait on the works to be finished before
stopping the application.
Commit: 78c90a2894e3db61fffb258b2a9f3fd6f8407f77
https://github.com/hibernate/hibernate-search/commit/78c90a2894e3db61fffb258b2a9f3fd6f8407f77
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessor.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/IndexWorkBuilder.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexer.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecution.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentIndexingWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkAggregator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentIndexingWork.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentWork.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessorTest.java
M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecutionTest.java
Log Message:
-----------
HSEARCH-3872 Expect all Elasticsearch batched works to be bulkable
This simplifies work execution as we no longer need the visitor pattern
that allowed us to detect whether a work was bulkable or not.
Commit: e281b7796479d574a8c0e272be45d8bef037445a
https://github.com/hibernate/hibernate-search/commit/e281b7796479d574a8c0e272be45d8bef037445a
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/ElasticsearchIndexManagerImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/AbstractElasticsearchWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSimpleWorkOrchestrator.java
R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorImplementor.java
Log Message:
-----------
HSEARCH-3872 Remove some orchestrator interfaces in the Elasticsearch backend
This abstraction is not necessary and just makes everything more complex.
Commit: bc52fa73e7c71588871f1551efa7e12d0627cd86
https://github.com/hibernate/hibernate-search/commit/bc52fa73e7c71588871f1551efa7e12d0627cd86
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkOrchestrator.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentIndexingWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java
A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexingWork.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentIndexingWork.java
Log Message:
-----------
HSEARCH-3872 Execute indexing works in parallel in the Elasticsearch backend
We use a hash of the document identifier to route works to a specific
queue, ensuring that works pertaining to the same document will be
executed in the same queue.
Commit: 581d00bb881288ca72e893c0b898226bdc9f08be
https://github.com/hibernate/hibernate-search/commit/581d00bb881288ca72e893c0b898226bdc9f08be
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/impl/ElasticsearchClientFactoryImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/spi/ElasticsearchClientFactory.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchLinkImpl.java
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/resources/impl/BackendThreads.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchClientFactoryImplIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchContentLengthIT.java
M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/testsupport/util/ElasticsearchClientSpy.java
M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/rule/TestElasticsearchClient.java
Log Message:
-----------
HSEARCH-3872 Use clearer thread names in the Elasticsearch backend
Commit: 853b435538fda3c790bd29b04ebb1d3f9be2a108
https://github.com/hibernate/hibernate-search/commit/853b435538fda3c790bd29b04ebb1d3f9be2a108
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java
Log Message:
-----------
HSEARCH-3872 Reduce the default max bulk size to 100
There seems to be a limit of around 250MB for all requests executing at
a given time, and Elasticsearch just rejects requests past that point.
With a parallelism of 10 and a max bulk size of 250, we can reach a
total of 2500 documents being indexed in parallel, which is apparently
enough to go over the 250MB limit in some tests with big documents
(around 100KB each, I suppose).
For the record, the test that highlighted the problem was
org.hibernate.search.integrationtest.backend.tck.search.query.SearchQueryTimeoutIT.
Commit: ab906fcd3b1a5c4b93c61ea8d9bf33daf149bba8
https://github.com/hibernate/hibernate-search/commit/ab906fcd3b1a5c4b93c61ea8d9bf33daf149bba8
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M engine/src/main/java/org/hibernate/search/engine/backend/orchestration/spi/BatchingExecutor.java
Log Message:
-----------
HSEARCH-3872 Log work processing in BatchingExecutor at debug level
Commit: 744734561e47019d3483acec9517dc54461c090a
https://github.com/hibernate/hibernate-search/commit/744734561e47019d3483acec9517dc54461c090a
Author: Yoann Rodière <yoann at hibernate.org>
Date: 2020-03-30 (Mon, 30 Mar 2020)
Changed paths:
M integrationtest/performance/backend/elasticsearch/src/main/resources/log4j.properties
Log Message:
-----------
HSEARCH-3872 Enable some logging in performance tests
Compare: https://github.com/hibernate/hibernate-search/compare/c1ddf64a7d26...744734561e47
More information about the hibernate-commits
mailing list