[hibernate-commits] [hibernate/hibernate-search] a3097e: HSEARCH-1886 Introduce OperationDispatcher, abstra...

Fri Apr 21 06:01:54 EDT 2017

  Branch: refs/heads/master
  Home:   https://github.com/hibernate/hibernate-search
  Commit: a3097e1b94884633b6fd72b9aa9e0a575ccb2a2a
      https://github.com/hibernate/hibernate-search/commit/a3097e1b94884633b6fd72b9aa9e0a575ccb2a2a
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/impl/BatchedQueueingProcessor.java
    A engine/src/main/java/org/hibernate/search/backend/impl/StreamingOperationDispatcher.java
    A engine/src/main/java/org/hibernate/search/backend/impl/TransactionalOperationDispatcher.java
    M engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java
    A engine/src/main/java/org/hibernate/search/backend/spi/OperationDispatcher.java

  Log Message:
  -----------
  HSEARCH-1886 Introduce OperationDispatcher, abstracting LuceneWork dispatching to IndexManagers

  Commit: 41aec302202a35b5a9e23d04923538a2e013d03f
      https://github.com/hibernate/hibernate-search/commit/41aec302202a35b5a9e23d04923538a2e013d03f
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M backends/jms/src/main/java/org/hibernate/search/backend/impl/jms/AbstractJMSHibernateSearchController.java
    M engine/src/main/java/org/hibernate/search/engine/impl/ImmutableSearchFactory.java
    M engine/src/main/java/org/hibernate/search/engine/impl/MutableSearchFactory.java
    M engine/src/main/java/org/hibernate/search/spi/SearchIntegrator.java
    M engine/src/main/java/org/hibernate/search/util/logging/impl/Log.java

  Log Message:
  -----------
  HSEARCH-1886 Fix the operation dispatching of the JMS backend for indexes with dynamic sharding

When dynamic sharding is enabled, we need to make every work go through
the sharding strategy, so that we are sure that:

1. The index manager targeted by each work is (lazily) created
2. If the index manager exists, its is aware that it should handle
the entity type of the given work (addContainedEntity has been called).
3. The internal state of the sharding strategy is properly updated
(it is aware that the given shard identifier exists and must be taken
into account when querying)

  Commit: 34861d2d7b73b24412ecd70b4d326934a1913a7b
      https://github.com/hibernate/hibernate-search/commit/34861d2d7b73b24412ecd70b4d326934a1913a7b
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsMasterMessageListener.java

  Log Message:
  -----------
  HSEARCH-1886 Fix the operation dispatching of the JGroups backend for indexes with dynamic sharding

When dynamic sharding is enabled, we need to make every work go through
the sharding strategy, so that we are sure that:

1. The index manager targeted by each work is (lazily) created
2. If the index manager exists, its is aware that it should handle
the entity type of the given work (addContainedEntity has been called).
3. The internal state of the sharding strategy is properly updated
(it is aware that the given shard identifier exists and must be taken
into account when querying)

  Commit: 7ca00bcad5f7c567f017f839a9a29d532a853b33
      https://github.com/hibernate/hibernate-search/commit/7ca00bcad5f7c567f017f839a9a29d532a853b33
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java
    M engine/src/main/java/org/hibernate/search/util/logging/impl/Log.java

  Log Message:
  -----------
  HSEARCH-1886 Remove the (useless) check over the Similarity of all the entities of a given index

The similarity is configured per index, so there is now way one could
configure different similarities for entities of a same index...

This is not really related to HSEARCH-1886, but since I am about to move
that code, we may as well make the move simpler.

  Commit: a8f48ebe7dc8d37c403743d98238c41a2144a759
      https://github.com/hibernate/hibernate-search/commit/a8f48ebe7dc8d37c403743d98238c41a2144a759
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    A engine/src/main/java/org/hibernate/search/engine/impl/AbstractMutableEntityIndexBinding.java
    R engine/src/main/java/org/hibernate/search/engine/impl/DefaultMutableEntityIndexBinding.java
    M engine/src/main/java/org/hibernate/search/engine/impl/DynamicShardingEntityIndexBinding.java
    R engine/src/main/java/org/hibernate/search/engine/impl/DynamicShardingStrategy.java
    R engine/src/main/java/org/hibernate/search/engine/impl/EntityIndexBindingFactory.java
    A engine/src/main/java/org/hibernate/search/engine/impl/NonDynamicShardingEntityIndexBinding.java
    A engine/src/main/java/org/hibernate/search/indexes/impl/DynamicShardingStrategy.java
    A engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerGroupHolder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java
    M engine/src/main/java/org/hibernate/search/store/IndexShardingStrategy.java

  Log Message:
  -----------
  HSEARCH-1886 Introduce IndexManagerGroupHolder, responsible for shards of a single index

We now have a component modeling a "group of indexes", i.e. a group
of shards for a same "logical" index, which will be useful to start
eagerly the JGroups listener on master, even if there are no index
managers yet.

Also, this reduces the complexity of dynamic sharding, and makes it
closer to non-dynamic sharding.

  Commit: 2de925d6a5fbef293e960201bc573c6cab65ab5d
      https://github.com/hibernate/hibernate-search/commit/2de925d6a5fbef293e960201bc573c6cab65ab5d
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/engine/impl/AbstractMutableEntityIndexBinding.java
    M engine/src/main/java/org/hibernate/search/engine/impl/DynamicShardingEntityIndexBinding.java
    M engine/src/main/java/org/hibernate/search/engine/impl/NonDynamicShardingEntityIndexBinding.java
    A engine/src/main/java/org/hibernate/search/indexes/impl/DynamicShardingEntityIndexBinder.java
    A engine/src/main/java/org/hibernate/search/indexes/impl/EntityIndexBinder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerGroupHolder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java
    A engine/src/main/java/org/hibernate/search/indexes/impl/NonDynamicShardingEntityIndexBinder.java
    M engine/src/main/java/org/hibernate/search/spi/SearchIntegratorBuilder.java

  Log Message:
  -----------
  HSEARCH-1886 Separate index initialiation from entity binding more clearly

1. Move the Similarity and IndexManagerType to the IndexManagerGroupHolder,
since this is a per-index-group setting.
2. Introduce the EntityBinder to encapsulate the process of adding an
entity to an index group. This will also allow different behavior
regarding backends depending on whether dynamic sharding is enabled
(see following commits).

  Commit: 0fc2d22c029ab985224c59f5d3a134ff003f84c8
      https://github.com/hibernate/hibernate-search/commit/0fc2d22c029ab985224c59f5d3a134ff003f84c8
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    A backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackend.java
    A backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsMasterBackend.java
    A backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsSlaveBackend.java
    M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/JGroupsReceivingMockBackend.java
    M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/SyncJGroupsBackendTest.java
    A backends/jms/src/main/java/org/hibernate/search/backend/jms/impl/JndiJMSBackend.java
    M documentation/src/main/asciidoc/architecture.asciidoc
    M documentation/src/main/asciidoc/configuration.asciidoc
    M engine/src/main/java/org/hibernate/search/backend/BackendFactory.java
    A engine/src/main/java/org/hibernate/search/backend/impl/InternalBackendFactory.java
    A engine/src/main/java/org/hibernate/search/backend/impl/LocalBackend.java
    M engine/src/main/java/org/hibernate/search/backend/impl/LocalBackendQueueProcessor.java
    A engine/src/main/java/org/hibernate/search/backend/impl/ReflectionBasedBackend.java
    A engine/src/main/java/org/hibernate/search/backend/impl/blackhole/BlackHoleBackend.java
    M engine/src/main/java/org/hibernate/search/backend/impl/blackhole/BlackHoleBackendQueueProcessor.java
    A engine/src/main/java/org/hibernate/search/backend/spi/Backend.java
    M engine/src/main/java/org/hibernate/search/backend/spi/BackendQueueProcessor.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/DynamicShardingEntityIndexBinder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/EntityIndexBinder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerGroupHolder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/IndexManagerHolder.java
    M engine/src/main/java/org/hibernate/search/indexes/impl/NonDynamicShardingEntityIndexBinder.java
    M engine/src/test/java/org/hibernate/search/test/backend/BackendQueueProcessorTest.java

  Log Message:
  -----------
  HSEARCH-1886 Introduce Backends, instantiated right when an index manager group is created

This will allow to start background services, such as JMS/JGroups
message consumers on the master nodes, even if there is no index manager
yet (as is the case with dynamic sharding when a *slave* triggers the
creation of a shard).

  Commit: 95a48c1d09801e83f1f688dc498382af3b3833b4
      https://github.com/hibernate/hibernate-search/commit/95a48c1d09801e83f1f688dc498382af3b3833b4
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M backends/jgroups/pom.xml
    M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackend.java
    M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackendQueueProcessor.java
    M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackendQueueTask.java
    M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/JGroupsBackendQueueProcessorTest.java
    M backends/jgroups/src/test/java/org/hibernate/search/backend/jgroups/impl/SyncJGroupsBackendTest.java
    M backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/slave/JGroupsSlaveTest.java

  Log Message:
  -----------
  HSEARCH-1886 Move JGroups initialization code from the queue processor to JGroupsBackend

This allow to start the JGroups message consumers on the master node,
even if there is no index manager yet (as is the case with dynamic
sharding when a *slave* triggers the creation of a shard).

  Commit: e62b27eeadb73c3c867866f4aeaed14fb77baa98
      https://github.com/hibernate/hibernate-search/commit/e62b27eeadb73c3c867866f4aeaed14fb77baa98
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M backends/jgroups/src/main/java/org/hibernate/search/backend/jgroups/impl/JGroupsMasterMessageListener.java

  Log Message:
  -----------
  HSEARCH-1886 Remove the reliance on the nodeSelector in JGroupsMasterMessageListener

Rationale:

 1. When using dynamic sharding, we may receive messages before the
target index manager has even been created, because the first entity to
be added to the index was created by a slave. In that case, there isn't
any node selector for this index (yet), and the message ends up being
swallowed.
 2. This check was only necessary to handle cases where the master just
became a slave, and a message was received by the old master just during
the switching. But in this case, the underlying queue processor is
supposed to have switched mode too, and *will* ultimately re-send the
message to the new master... So the check is simply redundant.

  Commit: dd2b6787482a645b73bd10316897960b259732e9
      https://github.com/hibernate/hibernate-search/commit/dd2b6787482a645b73bd10316897960b259732e9
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    M backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/master/JGroupsMasterTest.java
    M integrationtest/jms/src/test/java/org/hibernate/search/test/jms/master/JMSMasterTest.java

  Log Message:
  -----------
  HSEARCH-1886 Clean up JMSMasterTest/JGroupsMasterTest (a bit)

  Commit: f39ded2527d345d6ebd192a6d492c11f30d094a6
      https://github.com/hibernate/hibernate-search/commit/f39ded2527d345d6ebd192a6d492c11f30d094a6
  Author: Yoann Rodière <yoann at hibernate.org>
  Date:   2017-04-19 (Wed, 19 Apr 2017)

  Changed paths:
    A backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/master/JGroupsMasterDynamicShardingTest.java
    M backends/jgroups/src/test/java/org/hibernate/search/test/jgroups/master/JGroupsMasterTest.java
    A integrationtest/jms/src/test/java/org/hibernate/search/test/jms/master/JMSMasterDynamicShardingTest.java
    M integrationtest/jms/src/test/java/org/hibernate/search/test/jms/master/JMSMasterTest.java

  Log Message:
  -----------
  HSEARCH-1886 Test JMS/JGroups with dynamic sharding for newly created shards

  Commit: bb0e67026acff2766b118ceb1fa1ed4c326a38bf
      https://github.com/hibernate/hibernate-search/commit/bb0e67026acff2766b118ceb1fa1ed4c326a38bf
  Author: Sanne Grinovero <sanne at hibernate.org>
  Date:   2017-04-20 (Thu, 20 Apr 2017)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/impl/StreamingOperationDispatcher.java

  Log Message:
  -----------
  HSEARCH-1886 Simplify the new StreamingOperationDispatcher

  Commit: bb7eb02407ebca118c1f9642a1aeff9082326bba
      https://github.com/hibernate/hibernate-search/commit/bb7eb02407ebca118c1f9642a1aeff9082326bba
  Author: Sanne Grinovero <sanne at hibernate.org>
  Date:   2017-04-20 (Thu, 20 Apr 2017)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/spi/BackendQueueProcessor.java

  Log Message:
  -----------
  HSEARCH-1886 Mark BackendQueueProcessor#initialize as @Deprecated

Compare: https://github.com/hibernate/hibernate-search/compare/d2b79ca1bd3c...bb7eb02407eb