[hibernate-issues] [Hibernate-JIRA] Commented: (HSEARCH-1019) Profile mass indexer to reduce number of index commits

[Sanne Grinovero (JIRA)]

    [ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44829#comment-44829 ] 

Sanne Grinovero commented on HSEARCH-1019:
------------------------------------------

brief update: this is being more complex than expected, we have indeed some commit operations which should be omitted, but not so much and solving it I've improved performance by only 40%; there is a mayor regression on the cost of initializing a new Analyzer in Lucene as it's doing some class consistency check via reflection; usually these check are guarded by the assert keyword but not in this case. I'm looking if we can work around it at least in the MassIndexer, or if it's no longer the case in Lucene 3.5 and if we need to patch Lucene.

> Profile mass indexer to reduce number of index commits
> ------------------------------------------------------
>
>                 Key: HSEARCH-1019
>                 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-1019
>             Project: Hibernate Search
>          Issue Type: Improvement
>          Components: massindexer
>    Affects Versions: 4.0.0.Final
>            Reporter: Hardy Ferentschik
>            Assignee: Sanne Grinovero
>            Priority: Blocker
>             Fix For: 4.1
>
>
> We need to profile the new mass indexer design and try to reduce the number of index commits. Performance of the filesystem based directory declined to a bug fix in Lucene (https://issues.apache.org/jira/browse/LUCENE-3418). Trying to reduce the amount of index commits (and hence fsync calls) might alleviate the performance loss. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira