From do-not-reply at jboss.org Fri Sep 23 03:22:17 2011 Content-Type: multipart/mixed; boundary="===============8431273093526570571==" MIME-Version: 1.0 From: do-not-reply at jboss.org To: exo-jcr-commits at lists.jboss.org Subject: [exo-jcr-commits] exo-jcr SVN: r4943 - in jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main: resources/images and 1 other directory. Date: Fri, 23 Sep 2011 03:22:16 -0400 Message-ID: <201109230722.p8N7MG9m015923@svn01.web.mwc.hst.phx2.redhat.com> --===============8431273093526570571== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Author: nzamosenchuk Date: 2011-09-23 03:22:16 -0400 (Fri, 23 Sep 2011) New Revision: 4943 Added: jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main/resources/imag= es/JConsole.png Modified: jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main/docbook/en-US/= modules/jcr/query-handler-config.xml Log: EXOJCR-1493 : doc updated. Modified: jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main/docbook= /en-US/modules/jcr/query-handler-config.xml =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main/docbook/en-US= /modules/jcr/query-handler-config.xml 2011-09-23 07:19:54 UTC (rev 4942) +++ jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main/docbook/en-US= /modules/jcr/query-handler-config.xml 2011-09-23 07:22:16 UTC (rev 4943) @@ -205,6 +205,14 @@ RecoveryFilters, the mechanism of index synchronization for Local Index strategy. + + + async-reindexing + + Controls the process of re-indexing on JCR's startup.= If + flag set, indexing will be launched asynchronously, without + blocking the JCR. Default is "false". + @@ -334,7 +342,7 @@ <property name=3D"index-recove= ry-mode" value=3D"from-coordinator" /> = - There are couple implementations of filters: + There are couple implementations of filters: = @@ -352,6 +360,18 @@ = + org.exoplatform.services.jcr.impl.core.query.lucene.Conf= igurationPropertyRecoveryFilter + : return value of QueryHandler configuration property + "index-recovery-filter-forcereindexing". So index recovery can= be + controlled from configuration separately for each workspace. + I.e: + + <property name=3D"index-re= covery-filter" value=3D"org.exoplatform.services.jcr.impl.core.query.lucene= .ConfigurationPropertyRecoveryFilter" /> +<property name=3D"index-recovery-filter-forcereindexing" value=3D"true"= /> + + + + org.exoplatform.services.jcr.impl.core.query.lucene.DocN= umberRecoveryFilter : checks number of documents in index on coordinator side and self-side. Return true if differs. Advantage of this filter @@ -360,7 +380,7 @@ workspaces in each one. Only one is really heavily used in clu= ster : frontend/production. So using this filter will only reindex those workspaces that are really changed, without affecting ot= her - indexes thus greatly reducing startup time. + indexes thus greatly reducing startup time. @@ -403,6 +423,122 @@ =
+ Asynchronous reindexing + + Managing a big set of data using JCR in production environment + sometimes requires special operations with Indexes, stored on File Sys= tem. + One of those maintenance operations is a recreation of it. Also called + "re-indexing". There are various usecases when it's important to do. T= hey + include hardware faults, hard restarts, data-corruption, migrations and + JCR updates that brings new features related to index. Usually index + re-creation requested on server's startup or in runtime. + +
+ On startup indexing + + Common usecase for updating and re-creating the index is to st= op + the server and manually remove indexes for workspaces requiring it. = When + server will be started, missing indexes are automatically recovered = by + re-indexing. JCR Supports direct RDBMS re-indexing, that usually is + faster than ordinary and can be configured via QueryHandler parameter + "rdbms-reindexing" set to "true" (for more information please refer = to + "Query-handler configuration overview"). New feature to introduce is + asynchronous indexing on startup. Usually startup is blocked until + process is finished. Block can take any period of time, depending on + amount of data persisted in repositories. But this can be resolved by + using an asynchronous approaches of startup indexation. Saying brief= ly, + it performs all operations with index in background, without blocking + the repository. This is controlled by the value of "async-reindexing" + parameter in QueryHandler configuration. With asynchronous indexation + active, JCR starts with no active indexes present. Queries on JCR st= ill + can be executed without exceptions, but no results will be returned + until index creation completed. Checking index state is possible via + QueryManagerImpl: + + boolean online =3D ((QueryManage= rImpl)Workspace.getQueryManager()).getQueryHandeler().isOnline(); + + "OFFLINE" state means that index is currently re-creating. When + state changed, corresponding log event is printed. From the start of + background task index is switched to "OFFLINE", with following log e= vent + : + + [INFO] Setting index OFFLINE (repository/production[= system]). + + When process finished, two events are logged : + + [INFO] Created initial index for 143018 nodes (repos= itory/production[system]). +[INFO] Setting index ONLINE (repository/production[system]). + + Those two log lines indicates the end of process for workspace + given in brackets. Calling isOnline() as mentioned above, will also + return true. +
+ +
+ Hot Asynchronous Workspace Reindexing via JMX + + Some hard system faults, error during upgrades, migration issu= es + and some other factors may corrupt the index. Most likely end custom= ers + would like the production systems to fix index issues in run-time, + without delays and restarts. Current versions of JCR supports "Hot + Asynchronous Workspace Reindexing" feature. It allows end-user (Serv= ice + Administrator) to launch the process in background without stopping = or + blocking whole application by using any JMX-compatible console (see + screenshot below, "JConsole in action"). + + + + Server can continue working as expected while index = is + recreated. This depends on the flag "allow queries", passed via JMX + interface to reindex operation invocation. If the flag set, then + application continues working. But there is one critical limitation = the + end-users must be aware. Index is frozen while background task is + running. It meant that queries are performed on index present on the + moment of task startup and data written into repository after startup + won't be available through the search until process finished. Data a= dded + during re-indexation is also indexed, but will be available only when + task is done. Briefly, JCR makes the snapshot of indexes on asynch t= ask + startup and uses it for searches. When operation finished, stale ind= exes + replaced by newly created including newly added data. If flag "allow + queries" is set to false, then all queries will throw an exception w= hile + task is running. Current state can be acquired using the following J= MX + operation: + + + + getHotReindexingState() - returns information about latest + invocation: start time, if in progress or finish time if + done. + + +
+ +
+ Notices + + First of all, can't launch Hot re-indexing via JMX if index is + already in offline mode. It means that index is currently is invoked= in + some operations, like re-indexing at startup, copying in cluster to + another node or whatever. Another important this is Hot Asynchronous + Reindexing via JMX and "on startup" reindexing are completely differ= ent + features. So you can't get the state of startup reindexing using com= mand + getHotReindexingState in JMX interface, but there are some common JMX + operations: + + + + getIOMode - returns current index IO mode (READ_ONLY / + READ_WRITE), belongs to clustered configuration states; + + + + getState - returns current state: ONLINE / OFFLINE. + + +
+
+ +
Advanced tuning =
Added: jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/main/resources/= images/JConsole.png =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D (Binary files differ) Property changes on: jcr/trunk/exo.jcr.docs/exo.jcr.docs.developer/en/src/m= ain/resources/images/JConsole.png ___________________________________________________________________ Added: svn:mime-type + application/octet-stream --===============8431273093526570571==--