From hibernate-commits at lists.jboss.org Mon Sep 3 23:05:11 2007 Content-Type: multipart/mixed; boundary="===============1046205469618408281==" MIME-Version: 1.0 From: hibernate-commits at lists.jboss.org To: hibernate-commits at lists.jboss.org Subject: [hibernate-commits] Hibernate SVN: r13993 - search/trunk/doc/reference/en/modules. Date: Mon, 03 Sep 2007 23:05:11 -0400 Message-ID: --===============1046205469618408281== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Author: epbernard Date: 2007-09-03 23:05:10 -0400 (Mon, 03 Sep 2007) New Revision: 13993 Modified: search/trunk/doc/reference/en/modules/batchindex.xml search/trunk/doc/reference/en/modules/query.xml Log: documentaton on purge Modified: search/trunk/doc/reference/en/modules/batchindex.xml =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- search/trunk/doc/reference/en/modules/batchindex.xml 2007-09-04 02:57:3= 1 UTC (rev 13992) +++ search/trunk/doc/reference/en/modules/batchindex.xml 2007-09-04 03:05:1= 0 UTC (rev 13993) @@ -1,66 +1,60 @@ + Manual indexing = - Indexing +
+ Indexing = - It is sometimes useful to index an object event if this object is = not - inserted nor updated to the database. This is especially true when you w= ant - to build your index the first time. You can achieve that goal using the - FullTextSession . + It is sometimes useful to index an object event if this object is + not inserted nor updated to the database. This is especially true when= you + want to build your index the first time. You can achieve that goal usi= ng + the FullTextSession . = - FullTextSession fullTextSession =3D Search.createFullTex= tSession(session); + FullTextSession fullTextSession =3D Search.createFullT= extSession(session); Transaction tx =3D fullTextSession.beginTransaction(); for (Customer customer : customers) { fullTextSession.index(customer); } tx.commit(); //index are written at commit time = - For maximum efficiency, Hibernate Search batch index operations wh= ich - and execute them at commit time (Note: you don't need to use - org.hibernate.Transaction in a JTA - environment). + For maximum efficiency, Hibernate Search batch index operations + which and execute them at commit time (Note: you don't need to use + org.hibernate.Transaction in a JTA + environment). = - If you expect to index a lot of data, you need to be careful about - memory consumption: since all documents are kept in a queue until the - transaction commit, you can potentially face an OutOfMemoryException. + If you expect to index a lot of data, you need to be careful abo= ut + memory consumption: since all documents are kept in a queue until the + transaction commit, you can potentially face an + OutOfMemoryException. = - To avoid that, you can set up the - hibernate.search.worker.batch_size property to a - sensitive value: all index operations are queued until - batch_size is reached. Every time - batch_size is reached (or if the transaction is - committed), the queue is processed (freeing memory) and emptied. Be aware - that the changes cannot be rollbacked if the number of index elements go= es - beyond batch_size. Be also aware that the queue limit= s are - also applied on regular transparent indexing (and not only when - session.index() is used). That's why a sensitive - batch_size value is expected. + To avoid that, you can set up the + hibernate.search.worker.batch_size property to a + sensitive value: all index operations are queued until + batch_size is reached. Every time + batch_size is reached (or if the transaction is + committed), the queue is processed (freeing memory) and emptied. Be aw= are + that the changes cannot be rollbacked if the number of index elements = goes + beyond batch_size. Be also aware that the queue lim= its + are also applied on regular transparent indexing (and not only when + session.index() is used). That's why a sensitive + batch_size value is expected. = - Other parameters which also can effect indexing time and memory co= nsumption are = + Other parameters which also can effect indexing time and memory + consumption are + hibernate.search.[default|<indexname>].batch.merge_fact= or + , + hibernate.search.[default|<indexname>].batch.max_merge_= docs + and + hibernate.search.[default|<indexname>].batch.max_buffer= ed_docs + . These parameters are Lucene specific and Hibernate Search is just + passing these paramters through - see for more details. = - hibernate.search.[default|<indexname>].batch.merge_factor= + Here is an especially efficient way to index a given class (usef= ul + for index (re)initialization): = - , = - - hibernate.search.[default|<indexname>].batch.max_merge_do= cs - - and = - - hibernate.search.[default|<indexname>].batch.max_buffered= _docs - - . These parameters are Lucene specific and Hibernate Search is just pass= ing these paramters through - see = - - - - for more details. = - - Here is an especially efficient way to index a given class (useful= for - index (re)initialization): - - = - - fullTextSession.setFlushMode(FlushMode.MANUAL); + fullTextSession.setFlushMode(FlushMode.MANUAL); transaction =3D fullTextSession.beginTransaction(); //Scrollable results will avoid loading too many objects in memory ScrollableResults results =3D fullTextSession.createCriteria( Email.class = ).scroll( ScrollMode.FORWARD_ONLY ); @@ -72,11 +66,40 @@ } transaction.commit(); = - = + It is critical that batchSize in the previous + example matches the batch_size value described + previously. +
= - It is critical that batchSize in the previous - example matches the batch_size value described - previously. +
+ Purging = - = + It is equally possible to remove an entity or all entities of a + given type from a Lucene index without the need to physically remove t= hem + from the database. This operation is named purging and is done through= the + FullTextSession. + + FullTextSession fullTextSession =3D Search.createFullT= extSession(session); +Transaction tx =3D fullTextSession.beginTransaction(); +for (Customer customer : customers) { + fullTextSession.purge( Customer.class, custome= r.getId() ); +} +tx.commit(); //index are written at commit time + + Purging will remove the entity with the given id from the Lucene + index but will not touch the database. + + If you need to remove all entities of a given type, you can use = the + purgeAll method. + + FullTextSession fullTextSession =3D Search.createFullT= extSession(session); +Transaction tx =3D fullTextSession.beginTransaction(); +fullTextSession.purge( Customer.class ); +//optionally optimize the index +//fullTextSession.getSearchFactory().optimize( Customer.class ); +tx.commit(); //index are written at commit time + + It is recommended to optimize the index after such an + operation. +
\ No newline at end of file Modified: search/trunk/doc/reference/en/modules/query.xml =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- search/trunk/doc/reference/en/modules/query.xml 2007-09-04 02:57:31 UTC= (rev 13992) +++ search/trunk/doc/reference/en/modules/query.xml 2007-09-04 03:05:10 UTC= (rev 13993) @@ -85,10 +85,10 @@ FullTextSession fullTextSession =3D Search.createF= ullTextSession( session ); org.hibernate.Query fullTextQuery =3D fullTextSession.createFullTextQuery(= luceneQuery ); = - If not specified otherwise, the query will be executed again= st all indexed entities, - potentially returning all types of indexed classes. It is advised, - from a performance point of view, to restrict the returned - types: + If not specified otherwise, the query will be executed again= st + all indexed entities, potentially returning all types of indexed + classes. It is advised, from a performance point of view, to restr= ict + the returned types: = org.hibernate.Query fullTextQuery =3D fullTextSess= ion.createFullTextQuery( luceneQuery, Customer.class ); //or @@ -280,11 +280,18 @@ Search has to process all Lucene Hits elements (within the paginatio= n) when using list() , uniqueResult() and - iterate(). If you wish to minimize Lucene - document loading, scroll() is more appropri= ate, - Don't forget to close the ScrollableResults - object when you're done, since it keeps Lucene resources. Pagination= is - a preferred method over scrolling though. + iterate(). + + If you wish to minimize Lucene document loading, + scroll() is more appropriate. Don't forget = to + close the ScrollableResults object when you're + done, since it keeps Lucene resources. If you expect to use + scroll but wish to load objects in batch, y= ou + can use query.setFetchSize(): When an objec= t is + accessed, and if not already loaded, Hibernate Search will load the = next + fetchSize objects in one pass. + + Pagination is a preferred method over scrolling though. =
@@ -445,7 +452,7 @@ fullTextQuery.enableFullTextFilter("security").set= Parameter( "level", 5 ); = Each parameter name should have an associated setter on either t= he - filter or filter factory of the targeted named filter definition. + filter or filter factory of the targeted named filter definition. = public class SecurityFilterFactory { private Integer level; @@ -498,8 +505,8 @@ implementation to each of the parameters equals and hashcode methods. = - Why should filters be cached? There are two area where filter ca= ching - shines: + Why should filters be cached? There are two area where filter + caching shines: = --===============1046205469618408281==--