From hibernate-commits at lists.jboss.org Wed Dec 3 06:22:31 2008 Content-Type: multipart/mixed; boundary="===============3485256884941972907==" MIME-Version: 1.0 From: hibernate-commits at lists.jboss.org To: hibernate-commits at lists.jboss.org Subject: [hibernate-commits] Hibernate SVN: r15645 - search/trunk/doc/reference/en/modules. Date: Wed, 03 Dec 2008 06:22:31 -0500 Message-ID: --===============3485256884941972907== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Author: hardy.ferentschik Date: 2008-12-03 06:22:31 -0500 (Wed, 03 Dec 2008) New Revision: 15645 Modified: search/trunk/doc/reference/en/modules/configuration.xml Log: HSEARCH-303 Modified: search/trunk/doc/reference/en/modules/configuration.xml =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- search/trunk/doc/reference/en/modules/configuration.xml 2008-12-03 09:5= 4:33 UTC (rev 15644) +++ search/trunk/doc/reference/en/modules/configuration.xml 2008-12-03 11:2= 2:31 UTC (rev 15645) @@ -33,14 +33,17 @@ Directory configuration = Apache Lucene has a notion of Directory to st= ore - the index files. The Directory implementation can be customized, but - Lucene comes bundled with a file system - (FSDirectoryProvider) and a in memory - (RAMDirectoryProvider) implementation. Hibernate Se= arch - has the notion of DirectoryProvider that handles the - configuration and the initialization of the Lucene Directory. + the index files. The Directory implementation c= an + be customized, but Lucene comes bundled with a file system + (FSDirectoryProvider) and an in memory + (RAMDirectoryProvider) implementation. + DirectoryProviders are the Hibernate Search abstrac= tion + around a Lucene Directory and handle the + configuration and the initialization of the underlying Lucene resource= s. + shows the list of the + directory providers bundled with Hibernate Search. = - +
List of built-in Directory Providers = @@ -59,8 +62,7 @@ org.hibernate.search.store.FSDirectoryProvider = File system based directory. The directory used will be - <indexBase>/< @Indexed.index - > + <indexBase>/< indexName > = indexBase : Base directoryindexName: override @@ -142,7 +144,7 @@
= - If the built-in directory providers does not fit your needs, you= can + If the built-in directory providers do not fit your needs, you c= an write your own directory provider by implementing the org.hibernate.store.DirectoryProvider interface. @@ -158,17 +160,26 @@ hibernate.search.indexname.direct= ory_provider = - hibernate.search.default.directory_provider org.hibern= ate.search.store.FSDirectoryProvider + + Configuring directory providers + + hibernate.search.default.directory_provider org.hibe= rnate.search.store.FSDirectoryProvider hibernate.search.default.indexBase=3D/usr/lucene/indexes hibernate.search.Rules.directory_provider org.hibernate.search.store.RAMDi= rectoryProvider + = applied on = - @Indexed(index=3D"Status") + + Specifying the index name using the <literal>index</literal> + parameter of <classname>@Indexed</classname> + + @Indexed(index=3D"Status") public class Status { ... } = @Indexed(index=3D"Rules") public class Rule { ... } + = will create a file system directory in /usr/lucene/indexes/Status where the Status entit= ies @@ -176,7 +187,7 @@ Rules where Rule entities will be indexed. = You can easily define common rules like the directory provider a= nd - base directory, and override those default later on on a per index + base directory, and override those defaults later on on a per index basis. = Writing your own DirectoryProvider, you c= an @@ -189,10 +200,10 @@ In some extreme cases involving huge indexes (in size), it is necessary to split (shard) the indexing data of a given entity type in= to several Lucene indexes. This solution is not recommended until you rea= ch - significant index sizes and index update time are slowing down. The ma= in - drawback of index sharding is that searches will end up being slower s= ince - more files have to be opened for a single search. In other words don't= do - it until you have problems :) + significant index sizes and index update times are slowing the applica= tion + down. The main drawback of index sharding is that searches will end up + being slower since more files have to be opened for a single search. In + other words don't do it until you have problems :) = Despite this strong warning, Hibernate Search allows you to inde= x a given entity type into several sub indexes. Data is sharded into the @@ -201,8 +212,13 @@ strategy is enabled, unless the number of shards is configured. To configure the number of shards use the following property = - hibernate.search.<indexName>.sharding_strategy.n= br_of_shards 5 + + Enabling index sharding by specifying nbr_of_shards for a + specific index = + hibernate.search.<indexName>.sharding_strategy= .nbr_of_shards 5 + + This will use 5 different shards. = The default sharding strategy, when shards are set up, splits the @@ -212,8 +228,12 @@ IndexShardingStrategy and by setting the following property = - hibernate.search.<indexName>.sharding_strategy m= y.shardingstrategy.Implementation + + Specifying a custom sharding strategy = + hibernate.search.<indexName>.sharding_strategy= my.shardingstrategy.Implementation + + Each shard has an independent directory provider configuration as described in . The DirectoryProvider default name for the previous example are @@ -222,18 +242,23 @@ name of it's owning index followed by . (dot) and= its index number. = - hibernate.search.default.indexBase /usr/lucene/indexes + + Configuring the sharding configuration for an example entity + <classname>Animal</classname> = + hibernate.search.default.indexBase /usr/lucene/index= es + hibernate.search.Animal.sharding_strategy.nbr_of_shards 5 hibernate.search.Animal.directory_provider org.hibernate.search.store.FSDi= rectoryProvider hibernate.search.Animal.0.indexName Animal00 hibernate.search.Animal.3.indexBase /usr/lucene/sharded hibernate.search.Animal.3.indexName Animal03 + = This configuration uses the default id string hashing strategy a= nd shards the Animal index into 5 subindexes. All subindexes are - FSDirectoryProvider instances and the directory where each subindex is - stored is as followed: + FSDirectoryProvider instances and the directory + where each subindex is stored is as followed: = @@ -266,29 +291,27 @@
Sharing indexes (two entities into the same directory) = - It is possible to store more than one entity index information i= nto - a single Lucene index. - This is only presented here so that you know the option is available. There is really not much benefit in sharing indexes. = - There are actually two ways to accomplish merging an index: + It is technically possible to store the information of more than= one + entity into a single Lucene index. There are two ways to accomplish + this: = Configuring the underlying directory providers to point to t= he same physical index directory. In practice, you set the property hibernate.search.[fully qualified entity - name].indexName=3D(relative directory from indexBase) to= the - same value. + name].indexName to the same value. As an example let=E2= =80=99s use + the same index (directory) for the Furniture + and Animal entity. We just set + indexName for both entities to for example + =E2=80=9CAnimal=E2=80=9D. Both entities will then be stored in the= Animal + directory = - For example, let=E2=80=99s say that we want the Furniture en= tity to - actually be stored in the same directory as the Animal entity. We - would have the configuration setting set to =E2=80=9CAnimal=E2=80= =9D. Both entities - will then be stored in the Animal directory - hibernate.search.org.hibernate.search.= test.shards.Furniture.indexName =3D Aninal hibernate.search.org.hibernate.search.test.shards.Animal.indexName =3D Ani= nal @@ -296,10 +319,10 @@ Setting the @Indexed annotation=E2=80=99s index attribute of the entities you want = to - merge to the same value. - - If we wanted all Furniture instances to be indexed in the An= imal - index along with all instances of Animal we would specify + merge to the same value. If we again wanted all + Furniture instances to be indexed in the + Animal index along with all instances of + Animal we would specify @Indexed(index=3D=E2=80=9DAnimal=E2=80=9D) on both Animal and Furniture classes. @@ -311,7 +334,7 @@ Worker configuration = It is possible to refine how Hibernate Search interacts with Luc= ene - through the worker configuration. The work can be exected to the Lucene + through the worker configuration. The work can be executed to the Luce= ne directory or sent to a JMS queue for later processing. When processed = to the Lucene directory, the work can be processed synchronously or asynchronously to the transaction commit. @@ -399,14 +422,32 @@ This section describes in greater detail how to configure the Ma= ster / Slaves Hibernate Search architecture. = + + + + + + + + + + JMS Master/Slave architecture + overview. + +
Slave nodes = Every index update operation is sent to a JMS queue. Index que= ring operations are executed on a local index copy. = - ### slave configuration + + JMS Slave configuration = + ### slave configuration + ## DirectoryProvider # (remote) master location hibernate.search.default.sourceBase =3D /mnt/mastervolume/lucenedirs/maste= rcopy @@ -430,6 +471,7 @@ # hibernate.search.worker.execution =3D async # hibernate.search.worker.thread_pool.size =3D 2 # hibernate.search.worker.buffer_queue.max =3D 50 + = A file system local copy is recommended for faster search results. @@ -442,10 +484,13 @@ Master node = Every index update operation is taken from a JMS queue and - executed. The master index(es) is(are) copied on a regular basis. + executed. The master index is copied on a regular basis. = - ### master configuration + + JMS Master configuration = + ### master configuration + ## DirectoryProvider # (remote) master location where information is copied to hibernate.search.default.sourceBase =3D /mnt/mastervolume/lucenedirs/maste= rcopy @@ -461,15 +506,19 @@ = ## Backend configuration #Backend is the default lucene one + = The refresh period should be higher that the expected time copy. = In addition to the Hibernate Search framework configuration, a - Message Driven Bean should be written and set up to process index wo= rks - queue through JMS. + Message Driven Bean should be written and set up to process the index + works queue through JMS. = - @MessageDriven(activationConfig =3D { + + Message Driven Bean processing the indexing queue + + @MessageDriven(activationConfig =3D { @ActivationConfigProperty(propertyName=3D"destinationType", property= Value=3D"javax.jms.Queue"), @ActivationConfigProperty(propertyName=3D"destination", propertyValu= e=3D"queue/hibernatesearch"), @ActivationConfigProperty(propertyName=3D"DLQMaxResent", propertyVal= ue=3D"1") @@ -486,22 +535,17 @@ protected void cleanSessionIfNeeded(Session session) = } } + = - This example inherit the abstract JMS controller class availab= le - and implements a JavaEE 5 MDB. This implementation is given as an - example and, while most likely more complex, can be adjusted to make= use - of non Java EE Message Driven Beans. For more information about the + This example inherits from the abstract JMS controller class + available in the Hibernate Search source code and implements a JavaE= E 5 + MDB. This implementation is given as an example and, while most like= ly + be more complex, can be adjusted to make use of non Java EE Message + Driven Beans. For more information about the getSession() and cleanSessionIfNeeded(), please check AbstractJMSHibernateSearchController's javadoc. - - - Hibernate Search test suite makes use of JBoss Embedded to t= est - the JMS integration. It allows the unit test to run both the MDB - container and JBoss Messaging (JMS provider) in a standalone way - (marketed by some as "lightweight"). -
= @@ -550,17 +594,19 @@ Annotations or Hibernate EntityManager. If, for some reason you need= to disable it, set hibernate.search.autoregister_listeners to false. - Note that there is no performance runtime when the listeners are ena= bled - while no entity is indexable. + Note that there is no performance penalty when the listeners are ena= bled + even though no entities are indexed. = To enable Hibernate Search in Hibernate Core (ie. if you don't= use Hibernate Annotations), add the - FullTextIndexEventListener for the six Hibernate - events that occur after changes are executed to the database. Once - again, such a configuration is not useful with Hibernate Annotations= or - Hibernate EntityManager. + FullTextIndexEventListener for the following six + Hibernate events. = - <hibernate-configuration> + + Explicitly enabling Hibernate Search by configuring the + <classname>FullTextIndexEventListener</classname> + + <hibernate-configuration> <session-factory> ... <event type=3D"post-update"/> @@ -583,27 +629,7 @@ </event> </session-factory> </hibernate-configuration> - - Be sure to add the appropriate jar files in your classpath. Ch= eck - lib/README.TXT for the list of third party librar= ies. - A typical installation on top of Hibernate Annotations will add: - - - - hibernate-search.jar: the core - engine - - - - lucene-core.jar: Lucene core - engine - - - - solr-core.jar, solr-common.jar: - Additional analyzer infrastructure - - + =
--===============3485256884941972907==--