[infinispan-issues] [JBoss JIRA] (ISPN-3102) Analyse the performance of MassIndexingTest

[Adrian Nistor (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12809556#comment-12809556 ] 

Adrian Nistor commented on ISPN-3102:
-------------------------------------

A little experiment, remove SKIP_INDEXING at put() and also remove rebuildIndexes() -- the total time it takes to index at put is identical with the map-reduce based indexing. So at least we know the slowness of the indexing is not due to a bug in map-reduce. 

A second experiment, modify the test to run with ram directory provider or filesystem provider, the test is super fast. We can conclude the performance problem is definitely caused by the Infinispan directory provider. Tuning IndexWriter performance provided only minor improvements.

For now I propose to lower the number of entries to 200 from 2000 as this is a functional test, not a perf test. The performance of Infinispan directory provider should be re-addressed.
                
> Analyse the performance of MassIndexingTest
> -------------------------------------------
>
>                 Key: ISPN-3102
>                 URL: https://issues.jboss.org/browse/ISPN-3102
>             Project: Infinispan
>          Issue Type: Task
>            Reporter: Mircea Markus
>            Assignee: Adrian Nistor
>            Priority: Blocker
>             Fix For: 6.0.0.CR1
>
>
> MassIndexingTest runs on CI in 10 mins with TRACE on and 3 mins with TRACE off. According to Sanne this shouldn't take that long:
> {quote}
> sannegrinovero
> 11:50 mmarkus: the org.infinispan.query.distributed.MassIndexingTest test is adding 2000 entries to the cache. True that might be a bit excessive, still if it takes 10 minutes you have a problem in Infinispan core.
> mmarkus
> 11:52 sannegrinovero: each such write is indexed I take it?
> sannegrinovero
> 11:53 mmarkus: no: withFlags(Flag.SKIP_INDEXING)
> 11:54 mmarkus: but it's a 4 node DIST test using Map/Reduce.
> sannegrinovero
> 11:54 then the Map/Reduce job does some indexing but the indexing engine didn't change overnight 
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira