[hibernate-issues] [Hibernate-JIRA] Commented: (HSEARCH-1019) Profile mass indexer to reduce number of index commits

[Laurent Almeras (JIRA)]

    [ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=44944#comment-44944 ] 

Laurent Almeras commented on HSEARCH-1019:
------------------------------------------

Hi,

I wrote a minimal test on the case I report on push it on github at git://github.com/lalmeras/hsearch1019.git so that we can work on common metrics.

After clone, test can be launched. It creates 5000 simple entities and rebuild the index. Lucene dependency can be switched with profiles.

Lucene 3.3 : mvn -Plucene33 test
Lucene 3.4 : mvn -Plucene34 test

On my computer, I report :
Lucene 3.3 : 16s
Lucene 3.4 : 337s

My coworker, with a faster harddisk, reports :
Lucene 3.3 : 14s
Lucene 3.4 : 45s

The conclusion of this minimal test case is similar to our real case. I think I have a problem with my harddisk, but the performance hit with 3.4 exists. We both notice with Lucene 3.4 that harddisk is the bottleneck (poor cpu usage and iowait whereas with lucene 3.3 cpu is fully used).

> Profile mass indexer to reduce number of index commits
> ------------------------------------------------------
>
>                 Key: HSEARCH-1019
>                 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-1019
>             Project: Hibernate Search
>          Issue Type: Improvement
>          Components: massindexer
>    Affects Versions: 4.0.0.Final
>            Reporter: Hardy Ferentschik
>            Assignee: Sanne Grinovero
>            Priority: Blocker
>             Fix For: 4.1
>
>
> We need to profile the new mass indexer design and try to reduce the number of index commits. Performance of the filesystem based directory declined to a bug fix in Lucene (https://issues.apache.org/jira/browse/LUCENE-3418). Trying to reduce the amount of index commits (and hence fsync calls) might alleviate the performance loss. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira