Branch: refs/heads/master Home: https://github.com/hibernate/hibernate-search Commit: 1b16f546e5d5bccbb772171815074eb2169ae8ce https://github.com/hibernate/hibernate-search/commit/1b16f546e5d5bccbb772... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkBulker.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/work/ElasticsearchIndexingIT.java Log Message: ----------- HSEARCH-3872 Remove the minimum number of works in an Elasticsearch bulk Always bulk what can be bulked. The goal is to make work execution simpler. Commit: 6f109d1e229fded50ff64c124c50b5b6781e9171 https://github.com/hibernate/hibernate-search/commit/6f109d1e229fded50ff6... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkExecutionContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchImmutableWorkExecutionContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchRefreshableWorkExecutionContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkExecutionContextImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSequenceBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/query/impl/ElasticsearchSearchQueryImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/BulkWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ClearScrollWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CloseIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CountWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CreateIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteByQueryWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DropIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ElasticsearchWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ExplainWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/FlushWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/GetIndexMetadataWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/MergeSegmentsWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/OpenIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexAliasesWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexMappingWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexSettingsWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/RefreshWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ScrollWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/SearchWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/WaitForIndexStatusWorkBuilder.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableElasticsearchWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSimpleElasticsearchWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentElasticsearchWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ClearScrollWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CloseIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CountWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CreateIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteByQueryWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DropIndexWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchForwardingWorkExecutionContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkAggregator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkExecutionContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ExplainWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/FlushWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ForceMergeWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/GetIndexMetadataWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/NonBulkableElasticsearchWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/OpenIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexAliasesWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexSettingsWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexTypeMappingWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/RefreshWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ScrollWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SearchWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/WaitForIndexStatusWork.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/mapping/ElasticsearchFieldAttributesIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/mapping/ElasticsearchFieldTypesIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/mapping/ElasticsearchTypeNameMappingSchemaIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/work/ElasticsearchIndexingIT.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch5TestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch60TestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch63TestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch64TestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch67TestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/Elasticsearch7TestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/dialect/ElasticsearchTestDialect.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/rule/TestElasticsearchClient.java Log Message: ----------- HSEARCH-3872 Remove the ability to execute bulkable works outside of a bulk Just execute them in a bulk, always. It's simpler, and more importantly it allows us to get rid of the end-of-sequence refresh. That end-of-sequence refresh must disappear if we want to have multiple work queues and threads per index, sharded by document ID. In that case, works from a single work plan may be executed in different threads, thus we can no longer expect the refresh to be executed "just after the sequence". For now we'll execute refreshes as part of the bulk works, which may incur a significant performance cost, as we may execute multiple refreshes per index for each work plan. We may want to look into this in the next commits, but since forced refreshes already perform very badly, this is not a priority. Commit: 89e702f27a9a787d2a06a9778334447f9095c621 https://github.com/hibernate/hibernate-search/commit/89e702f27a9a787d2a06... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/Elasticsearch67WorkBuilderFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/Elasticsearch7WorkBuilderFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/ElasticsearchWorkBuilderFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexer.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java Log Message: ----------- HSEARCH-3872 Do not URL-encode document identifiers for Elasticsearch indexing works Now that works are always bulked, we do not need URL-encoding anymore. Commit: 6f2f84dc69ac76191e69541dcc9e482fd1fecedc https://github.com/hibernate/hibernate-search/commit/6f2f84dc69ac76191e69... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkBulker.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSequenceBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/Elasticsearch7WorkBuilderFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/factory/impl/ElasticsearchWorkBuilderFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/BulkWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ClearScrollWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CloseIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CountWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/CreateIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteByQueryWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DropIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ExplainWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/FlushWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/GetIndexMetadataWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/IndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/MergeSegmentsWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/OpenIndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexAliasesWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexMappingWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/PutIndexSettingsWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/RefreshWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/ScrollWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/SearchWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/WaitForIndexStatusWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSet.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableElasticsearchWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentElasticsearchWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkableElasticsearchWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkableWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ClearScrollWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CloseIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CountWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/CreateIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteByQueryWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DropIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkAggregator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ExplainWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/FlushWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ForceMergeWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/GetIndexMetadataWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/NonBulkableElasticsearchWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/NonBulkableWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/OpenIndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexAliasesWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexSettingsWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/PutIndexTypeMappingWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/RefreshWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ScrollWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SearchWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentElasticsearchWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/WaitForIndexStatusWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/result/impl/BulkResultItemExtractor.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWorkTest.java Log Message: ----------- HSEARCH-3872 Remove an unnecessary "Elasticsearch" component in the class name of Elasticsearch works Commit: a062bef7e3de32bd86e5960a43046680c8007b7b https://github.com/hibernate/hibernate-search/commit/a062bef7e3de32bd86e5... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/IndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSet.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentWork.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java Log Message: ----------- HSEARCH-3872 Give a common return type to all works implementing SingleDocumentWriteWork This will be useful when manipulating futures and single-document works in the next commits. Commit: 9cd4135b36bd04a0ff6d2d99c85e60a47cae25a0 https://github.com/hibernate/hibernate-search/commit/9cd4135b36bd04a0ff6d... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/logging/impl/Log.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java Log Message: ----------- HSEARCH-3872 Remove the requirement for works within a write plan to be skipped when a previous work failed Because: 1. It's not necessary. The only works added to a write plan are add, update, delete, and we expect to be able to execute these works even if a previous add/update/delete failed. 2. It won't be possible anymore once we organize work execution in multiple queues sharded by ID, where we will lose the information "this work belongs to the same write plan as this other work". Commit: fb00b50300c07832a23277e42ed1a36d4c1120c3 https://github.com/hibernate/hibernate-search/commit/fb00b50300c07832a232... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSingleWorkSet.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecution.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSet.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java A backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecutionTest.java R backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexingPlanWorkSetTest.java Log Message: ----------- HSEARCH-3872 Remove the concept of "work set" from Elasticsearch work processors Only ever process works one by one. This is necessary if we want to have multiple work queues and threads per index, sharded by document ID. In that case, works from a single work plan may be executed in different threads, thus we can no longer pass "work sets" to the processors. We need something to be notified that all works have been executed, and to act accordingly. Commit: 3d562d20ce1f19cecfe3be261accd0f10c010668 https://github.com/hibernate/hibernate-search/commit/3d562d20ce1f19cecfe3... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkBulker.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java Log Message: ----------- HSEARCH-3872 Add works to the sequence automatically in ElasticsearchWorkBulker In the past this was not possible and we had to expose addWorksToSequence() in order for clients to add the works explicitly, but with the removals of the past few commits, we can simplify that. Commit: 9a982b0e59c7a63a68fb73d95da18b3a080d841e https://github.com/hibernate/hibernate-search/commit/9a982b0e59c7a63a68fb... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulker.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSequenceBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/result/impl/BulkResult.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/result/impl/BulkResultItemExtractor.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkBulkerTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchDefaultWorkSequenceBuilderTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/impl/BulkWorkTest.java Log Message: ----------- HSEARCH-3872 Add bulk extraction through a single method call in ElasticsearchWorkSequenceBuilder In the past this was not possible and we had to expose an intermediary object to add extractions (ElasticsearchWorkSequenceBuilder.BulkResultExtractionStep), but with the removals of the past few commits, we can simplify that. Commit: aa1c39fe5fcd9ff650c02f87dd2e34c5d20dfdaa https://github.com/hibernate/hibernate-search/commit/aa1c39fe5fcd9ff650c0... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/AbstractElasticsearchWorkOrchestrator.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSingleWorkSet.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkSet.java Log Message: ----------- HSEARCH-3872 Remove the concept of "work set" in the Elasticsearch backend Work sets no longer make sense, at least during work orchestration, since we submit works one by one without any information about the relationship between works. Commit: 0ef199359633b090287bb3e78158c183bb36ba6f https://github.com/hibernate/hibernate-search/commit/0ef199359633b090287b... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/impl/ElasticsearchClientFactoryImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/impl/ElasticsearchClientImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/spi/ElasticsearchClientFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchLinkImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/IndexManagerBackendContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/resources/impl/BackendThreads.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchClientFactoryImplIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchContentLengthIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/testsupport/util/ElasticsearchClientSpy.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/rule/TestElasticsearchClient.java Log Message: ----------- HSEARCH-3872 Use a single thread pool for all Elasticsearch indexes It may reduce performance when there are many indexes, but it brings the number of threads to a more reasonable amount in that case, and it'll allow us to introduce sharding so as to perform better when a single index is getting hit by many write requests. Commit: 7659094b0b737a71a3c6ea64d8cbdd2d24ec952d https://github.com/hibernate/hibernate-search/commit/7659094b0b737a71a3c6... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/writer/impl/IndexWriterDelegatorImpl.java M engine/src/main/java/org/hibernate/search/engine/backend/orchestration/spi/BatchingExecutor.java M engine/src/main/java/org/hibernate/search/engine/backend/orchestration/spi/SingletonTask.java Log Message: ----------- HSEARCH-3872 Do not block when waiting for work completion in SingletonTask/BatchingExecutor So that the thread pool can be used for other tasks. This is especially important for the Elasticsearch backend, where we execute work asynchronously with a parallelism equal to the number of connections, and have potentially much more connections available than the number of threads in the pool. Commit: 11a7019eabdd7f5617512fc838e1f327cf2fdc21 https://github.com/hibernate/hibernate-search/commit/11a7019eabdd7f561751... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/ElasticsearchIndexManagerImpl.java Log Message: ----------- HSEARCH-3872 Execute mass indexing works in the serial orchestrator in the Elasticsearch backend Because we need to preserve the order of these works relative to indexing plan works, as much as possible. This may decrease performance of mass indexing on Elasticsearch, but we'll bring it back up by sharding work queues for indexing in the next commits. Commit: f150c2ee1c317b32f2535c4f4036be18a8816780 https://github.com/hibernate/hibernate-search/commit/f150c2ee1c317b32f253... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/ElasticsearchIndexManagerImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/IndexManagerBackendContext.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/AbstractElasticsearchWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkOrchestrator.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessor.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkOrchestrator.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessor.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSimpleWorkOrchestrator.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorImplementor.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorProvider.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/schema/management/impl/ElasticsearchIndexSchemaManager.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/schema/management/impl/ElasticsearchSchemaAccessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/query/impl/ElasticsearchSearchQueryBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/query/impl/ElasticsearchSearchQueryImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexer.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecution.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexWorkspace.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/WorkExecutionBackendContext.java A backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessorTest.java R backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchParallelWorkProcessorTest.java R backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecutionTest.java Log Message: ----------- HSEARCH-3872 Do not batch Elasticsearch works when executing them in parallel No need for a batching orchestrator, as we don't ever bulk these works and we always expect the user to wait on the works to be finished before stopping the application. Commit: 78c90a2894e3db61fffb258b2a9f3fd6f8407f77 https://github.com/hibernate/hibernate-search/commit/78c90a2894e3db61fffb... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessor.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/DeleteWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/builder/impl/IndexWorkBuilder.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexer.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlan.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecution.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractNonBulkableWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentIndexingWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/ElasticsearchWorkAggregator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentIndexingWork.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentWork.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWorkProcessorTest.java M backend/elasticsearch/src/test/java/org/hibernate/search/backend/elasticsearch/work/execution/impl/ElasticsearchIndexIndexingPlanExecutionTest.java Log Message: ----------- HSEARCH-3872 Expect all Elasticsearch batched works to be bulkable This simplifies work execution as we no longer need the visitor pattern that allowed us to detect whether a work was bulkable or not. Commit: e281b7796479d574a8c0e272be45d8bef037445a https://github.com/hibernate/hibernate-search/commit/e281b7796479d574a8c0... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchBackendImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/index/impl/ElasticsearchIndexManagerImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/AbstractElasticsearchWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSimpleWorkOrchestrator.java R backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchWorkOrchestratorImplementor.java Log Message: ----------- HSEARCH-3872 Remove some orchestrator interfaces in the Elasticsearch backend This abstraction is not necessary and just makes everything more complex. Commit: bc52fa73e7c71588871f1551efa7e12d0627cd86 https://github.com/hibernate/hibernate-search/commit/bc52fa73e7c71588871f... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchedWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchSerialWorkOrchestrator.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/AbstractSingleDocumentIndexingWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/DeleteWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexWork.java A backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/IndexingWork.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/work/impl/SingleDocumentIndexingWork.java Log Message: ----------- HSEARCH-3872 Execute indexing works in parallel in the Elasticsearch backend We use a hash of the document identifier to route works to a specific queue, ensuring that works pertaining to the same document will be executed in the same queue. Commit: 581d00bb881288ca72e893c0b898226bdc9f08be https://github.com/hibernate/hibernate-search/commit/581d00bb881288ca72e8... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/impl/ElasticsearchClientFactoryImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/client/spi/ElasticsearchClientFactory.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/impl/ElasticsearchLinkImpl.java M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/resources/impl/BackendThreads.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchClientFactoryImplIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/client/ElasticsearchContentLengthIT.java M integrationtest/backend/elasticsearch/src/test/java/org/hibernate/search/integrationtest/backend/elasticsearch/testsupport/util/ElasticsearchClientSpy.java M util/internal/integrationtest/backend/elasticsearch/src/main/java/org/hibernate/search/util/impl/integrationtest/backend/elasticsearch/rule/TestElasticsearchClient.java Log Message: ----------- HSEARCH-3872 Use clearer thread names in the Elasticsearch backend Commit: 853b435538fda3c790bd29b04ebb1d3f9be2a108 https://github.com/hibernate/hibernate-search/commit/853b435538fda3c790bd... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/orchestration/impl/ElasticsearchBatchingWorkOrchestrator.java Log Message: ----------- HSEARCH-3872 Reduce the default max bulk size to 100 There seems to be a limit of around 250MB for all requests executing at a given time, and Elasticsearch just rejects requests past that point. With a parallelism of 10 and a max bulk size of 250, we can reach a total of 2500 documents being indexed in parallel, which is apparently enough to go over the 250MB limit in some tests with big documents (around 100KB each, I suppose). For the record, the test that highlighted the problem was org.hibernate.search.integrationtest.backend.tck.search.query.SearchQueryTimeoutIT. Commit: ab906fcd3b1a5c4b93c61ea8d9bf33daf149bba8 https://github.com/hibernate/hibernate-search/commit/ab906fcd3b1a5c4b93c6... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M engine/src/main/java/org/hibernate/search/engine/backend/orchestration/spi/BatchingExecutor.java Log Message: ----------- HSEARCH-3872 Log work processing in BatchingExecutor at debug level Commit: 744734561e47019d3483acec9517dc54461c090a https://github.com/hibernate/hibernate-search/commit/744734561e47019d3483... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-03-30 (Mon, 30 Mar 2020) Changed paths: M integrationtest/performance/backend/elasticsearch/src/main/resources/log4j.properties Log Message: ----------- HSEARCH-3872 Enable some logging in performance tests Compare: https://github.com/hibernate/hibernate-search/compare/c1ddf64a7d26...7447...