[infinispan-issues] [JBoss JIRA] (ISPN-11512) The mass indexer should iterate over the cache only once

[Dan Berindei (Jira)]

Dan Berindei created ISPN-11512:
-----------------------------------

             Summary: The mass indexer should iterate over the cache only once
                 Key: ISPN-11512
                 URL: https://issues.redhat.com/browse/ISPN-11512
             Project: Infinispan
          Issue Type: Enhancement
          Components: Indexing
    Affects Versions: 11.0.0.Dev03, 10.1.5.Final
            Reporter: Dan Berindei
            Assignee: Gustavo Fernandes
             Fix For: 11.0.0.Final


{{DistributedExecutorMassIndexer}} starts an {{IndexWorker}} for each indexed type and submits them to all the nodes in parallel. Each {{IndexWorker}} runs a blocking iteration over the cache, and when the cache has a store, that iteration is very expensive.

Each iteration will load and deserialize all the entries in the store. Most stores don't implement {{AbstractSegmentedStoreConfiguration}}, so Infinispan wraps them in a {{ComposedSegmentedLoadWriteStore}}, which iterates over each segment in parallel on the persistence/blocking executor. Since the default number of segments is 256 and the persistence/blocking executor has 4*cpu_count/150 max threads, it doesn't take a lot of parallel iterations to fill the blocking executor's threads and prevent the cache from doing other, more urgent, work.



--
This message was sent by Atlassian Jira
(v7.13.8#713008)