[hibernate-commits] [hibernate/hibernate-search] a3097e: HSEARCH-1886 Introduce OperationDispatcher, abstra...

Branch: refs/heads/master Home: https://github.com/hibernate/hibernate-search Commit: a3097e1b94884633b6fd72b9aa9e0a575ccb2a2a https://github.com/hibernate/hibernate-search/commit/a3097e1b94884633b6fd... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M engine/src/main/java/org/hibernate/search/backend/impl/BatchedQueueingProcessor.java A engine/src/main/java/org/hibernate/search/backend/impl/StreamingOperationDispatcher.java A engine/src/main/java/org/hibernate/search/backend/impl/TransactionalOperationDispatcher.java M engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java A engine/src/main/java/org/hibernate/search/backend/spi/OperationDispatcher.java Log Message: ----------- HSEARCH-1886 Introduce OperationDispatcher, abstracting LuceneWork dispatching to IndexManagers Commit: 41aec302202a35b5a9e23d04923538a2e013d03f https://github.com/hibernate/hibernate-search/commit/41aec302202a35b5a9e2... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M backends/jms/src/main/java/org/hibernate/search/backend/impl/jms/AbstractJMSHibernateSearchController.java M engine/src/main/java/org/hibernate/search/engine/impl/ImmutableSearchFactory.java M engine/src/main/java/org/hibernate/search/engine/impl/MutableSearchFactory.java M engine/src/main/java/org/hibernate/search/spi/SearchIntegrator.java M engine/src/main/java/org/hibernate/search/util/logging/impl/Log.java Log Message: ----------- HSEARCH-1886 Fix the operation dispatching of the JMS backend for indexes with dynamic sharding When dynamic sharding is enabled, we need to make every work go through the sharding strategy, so that we are sure that: 1. The index manager targeted by each work is (lazily) created 2. If the index manager exists, its is aware that it should handle the entity type of the given work (addContainedEntity has been called). 3. The internal state of the sharding strategy is properly updated (it is aware that the given shard identifier exists and must be taken into account when querying) Commit: 34861d2d7b73b24412ecd70b4d326934a1913a7b https://github.com/hibernate/hibernate-search/commit/34861d2d7b73b24412ec... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsMasterMessageListener.java Log Message: ----------- HSEARCH-1886 Fix the operation dispatching of the JGroups backend for indexes with dynamic sharding When dynamic sharding is enabled, we need to make every work go through the sharding strategy, so that we are sure that: 1. The index manager targeted by each work is (lazily) created 2. If the index manager exists, its is aware that it should handle the entity type of the given work (addContainedEntity has been called). 3. The internal state of the sharding strategy is properly updated (it is aware that the given shard identifier exists and must be taken into account when querying) Commit: 7ca00bcad5f7c567f017f839a9a29d532a853b33 https://github.com/hibernate/hibernate-search/commit/7ca00bcad5f7c567f017... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java M engine/src/main/java/org/hibernate/search/util/logging/impl/Log.java Log Message: ----------- HSEARCH-1886 Remove the (useless) check over the Similarity of all the entities of a given index The similarity is configured per index, so there is now way one could configure different similarities for entities of a same index... This is not really related to HSEARCH-1886, but since I am about to move that code, we may as well make the move simpler. Commit: a8f48ebe7dc8d37c403743d98238c41a2144a759 https://github.com/hibernate/hibernate-search/commit/a8f48ebe7dc8d37c4037... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: A engine/src/main/java/org/hibernate/search/engine/impl/AbstractMutableEntityIndexBinding.java R engine/src/main/java/org/hibernate/search/engine/impl/DefaultMutableEntityIndexBinding.java M engine/src/main/java/org/hibernate/search/engine/impl/DynamicShardingEntityIndexBinding.java R engine/src/main/java/org/hibernate/search/engine/impl/DynamicShardingStrategy.java R engine/src/main/java/org/hibernate/search/engine/impl/EntityIndexBindingFactory.java A engine/src/main/java/org/hibernate/search/engine/impl/NonDynamicShardingEntityIndexBinding.java A engine/src/main/java/org/hibernate/search/indexes/impl/DynamicShardingStrategy.java A engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerGroupHolder.java M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java M engine/src/main/java/org/hibernate/search/store/IndexShardingStrategy.java Log Message: ----------- HSEARCH-1886 Introduce IndexManagerGroupHolder, responsible for shards of a single index We now have a component modeling a "group of indexes", i.e. a group of shards for a same "logical" index, which will be useful to start eagerly the JGroups listener on master, even if there are no index managers yet. Also, this reduces the complexity of dynamic sharding, and makes it closer to non-dynamic sharding. Commit: 2de925d6a5fbef293e960201bc573c6cab65ab5d https://github.com/hibernate/hibernate-search/commit/2de925d6a5fbef293e96... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M engine/src/main/java/org/hibernate/search/engine/impl/AbstractMutableEntityIndexBinding.java M engine/src/main/java/org/hibernate/search/engine/impl/DynamicShardingEntityIndexBinding.java M engine/src/main/java/org/hibernate/search/engine/impl/NonDynamicShardingEntityIndexBinding.java A engine/src/main/java/org/hibernate/search/indexes/impl/DynamicShardingEntityIndexBinder.java A engine/src/main/java/org/hibernate/search/indexes/impl/EntityIndexBinder.java M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerGroupHolder.java M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java A engine/src/main/java/org/hibernate/search/indexes/impl/NonDynamicShardingEntityIndexBinder.java M engine/src/main/java/org/hibernate/search/spi/SearchIntegratorBuilder.java Log Message: ----------- HSEARCH-1886 Separate index initialiation from entity binding more clearly 1. Move the Similarity and IndexManagerType to the IndexManagerGroupHolder, since this is a per-index-group setting. 2. Introduce the EntityBinder to encapsulate the process of adding an entity to an index group. This will also allow different behavior regarding backends depending on whether dynamic sharding is enabled (see following commits). Commit: 0fc2d22c029ab985224c59f5d3a134ff003f84c8 https://github.com/hibernate/hibernate-search/commit/0fc2d22c029ab985224c... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: A backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackend.java A backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsMasterBackend.java A backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsSlaveBackend.java M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/JGroupsReceivingMockBackend.java M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/SyncJGroupsBackendTest.java A backends/jms/src/main/java/org/hibernate/search/backend/jms/impl/JndiJMSBackend.java M documentation/src/main/asciidoc/architecture.asciidoc M documentation/src/main/asciidoc/configuration.asciidoc M engine/src/main/java/org/hibernate/search/backend/BackendFactory.java A engine/src/main/java/org/hibernate/search/backend/impl/InternalBackendFactory.java A engine/src/main/java/org/hibernate/search/backend/impl/LocalBackend.java M engine/src/main/java/org/hibernate/search/backend/impl/LocalBackendQueueProcessor.java A engine/src/main/java/org/hibernate/search/backend/impl/ReflectionBasedBackend.java A engine/src/main/java/org/hibernate/search/backend/impl/blackhole/BlackHoleBackend.java M engine/src/main/java/org/hibernate/search/backend/impl/blackhole/BlackHoleBackendQueueProcessor.java A engine/src/main/java/org/hibernate/search/backend/spi/Backend.java M engine/src/main/java/org/hibernate/search/backend/spi/BackendQueueProcessor.java M engine/src/main/java/org/hibernate/search/indexes/impl/DynamicShardingEntityIndexBinder.java M engine/src/main/java/org/hibernate/search/indexes/impl/EntityIndexBinder.java M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerGroupHolder.java M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java M engine/src/main/java/org/hibernate/search/indexes/impl/NonDynamicShardingEntityIndexBinder.java M engine/src/test/java/org/hibernate/search/test/backend/BackendQueueProcessorTest.java Log Message: ----------- HSEARCH-1886 Introduce Backends, instantiated right when an index manager group is created This will allow to start background services, such as JMS/JGroups message consumers on the master nodes, even if there is no index manager yet (as is the case with dynamic sharding when a *slave* triggers the creation of a shard). Commit: 95a48c1d09801e83f1f688dc498382af3b3833b4 https://github.com/hibernate/hibernate-search/commit/95a48c1d09801e83f1f6... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M backends/jgroups/pom.xml M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackend.java M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackendQueueProcessor.java M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackendQueueTask.java M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackendQueueProcessorTest.java M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/SyncJGroupsBackendTest.java M backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/slave/JGroupsSlaveTest.java Log Message: ----------- HSEARCH-1886 Move JGroups initialization code from the queue processor to JGroupsBackend This allow to start the JGroups message consumers on the master node, even if there is no index manager yet (as is the case with dynamic sharding when a *slave* triggers the creation of a shard). Commit: e62b27eeadb73c3c867866f4aeaed14fb77baa98 https://github.com/hibernate/hibernate-search/commit/e62b27eeadb73c3c8678... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsMasterMessageListener.java Log Message: ----------- HSEARCH-1886 Remove the reliance on the nodeSelector in JGroupsMasterMessageListener Rationale: 1. When using dynamic sharding, we may receive messages before the target index manager has even been created, because the first entity to be added to the index was created by a slave. In that case, there isn't any node selector for this index (yet), and the message ends up being swallowed. 2. This check was only necessary to handle cases where the master just became a slave, and a message was received by the old master just during the switching. But in this case, the underlying queue processor is supposed to have switched mode too, and *will* ultimately re-send the message to the new master... So the check is simply redundant. Commit: dd2b6787482a645b73bd10316897960b259732e9 https://github.com/hibernate/hibernate-search/commit/dd2b6787482a645b73bd... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: M backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/master/JGroupsMasterTest.java M integrationtest/jms/src/test/java/org/hibernate/search/test/jms/master/JMSMasterTest.java Log Message: ----------- HSEARCH-1886 Clean up JMSMasterTest/JGroupsMasterTest (a bit) Commit: f39ded2527d345d6ebd192a6d492c11f30d094a6 https://github.com/hibernate/hibernate-search/commit/f39ded2527d345d6ebd1... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2017-04-19 (Wed, 19 Apr 2017) Changed paths: A backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/master/JGroupsMasterDynamicShardingTest.java M backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/master/JGroupsMasterTest.java A integrationtest/jms/src/test/java/org/hibernate/search/test/jms/master/JMSMasterDynamicShardingTest.java M integrationtest/jms/src/test/java/org/hibernate/search/test/jms/master/JMSMasterTest.java Log Message: ----------- HSEARCH-1886 Test JMS/JGroups with dynamic sharding for newly created shards Commit: bb0e67026acff2766b118ceb1fa1ed4c326a38bf https://github.com/hibernate/hibernate-search/commit/bb0e67026acff2766b11... Author: Sanne Grinovero <sanne(a)hibernate.org&gt; Date: 2017-04-20 (Thu, 20 Apr 2017) Changed paths: M engine/src/main/java/org/hibernate/search/backend/impl/StreamingOperationDispatcher.java Log Message: ----------- HSEARCH-1886 Simplify the new StreamingOperationDispatcher Commit: bb7eb02407ebca118c1f9642a1aeff9082326bba https://github.com/hibernate/hibernate-search/commit/bb7eb02407ebca118c1f... Author: Sanne Grinovero <sanne(a)hibernate.org&gt; Date: 2017-04-20 (Thu, 20 Apr 2017) Changed paths: M engine/src/main/java/org/hibernate/search/backend/spi/BackendQueueProcessor.java Log Message: ----------- HSEARCH-1886 Mark BackendQueueProcessor#initialize as @Deprecated Compare: https://github.com/hibernate/hibernate-search/compare/d2b79ca1bd3c...bb7e...