[infinispan-issues] [JBoss JIRA] (ISPN-11512) The mass indexer should iterate over the cache only once

[Gustavo Fernandes (Jira)]

     [ https://issues.redhat.com/browse/ISPN-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gustavo Fernandes updated ISPN-11512:
-------------------------------------
    Status: Open  (was: New)


> The mass indexer should iterate over the cache only once
> --------------------------------------------------------
>
>                 Key: ISPN-11512
>                 URL: https://issues.redhat.com/browse/ISPN-11512
>             Project: Infinispan
>          Issue Type: Enhancement
>          Components: Indexing
>    Affects Versions: 10.1.5.Final, 11.0.0.Dev03
>            Reporter: Dan Berindei
>            Assignee: Gustavo Fernandes
>            Priority: Major
>             Fix For: 11.0.0.Final
>
>
> {{DistributedExecutorMassIndexer}} starts an {{IndexWorker}} for each indexed type and submits them to all the nodes in parallel. Each {{IndexWorker}} runs a blocking iteration over the cache, and when the cache has a store, that iteration is very expensive.
> Each iteration will load and deserialize all the entries in the store. Most stores don't implement {{AbstractSegmentedStoreConfiguration}}, so Infinispan wraps them in a {{ComposedSegmentedLoadWriteStore}}, which iterates over each segment in parallel on the persistence/blocking executor. Since the default number of segments is 256 and the persistence/blocking executor has 4*cpu_count/150 max threads, it doesn't take a lot of parallel iterations to fill the blocking executor's threads and prevent the cache from doing other, more urgent, work.



--
This message was sent by Atlassian Jira
(v7.13.8#713008)