[JBoss JIRA] (ISPN-4847) Improve indexing performance
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4847?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-4847:
---------------------------------------
Not sure if ISPN-4596 belongs on this list? If we want to add all improvements we see feasible, then we might need to split this in multiple milestones.
> Improve indexing performance
> ----------------------------
>
> Key: ISPN-4847
> URL: https://issues.jboss.org/browse/ISPN-4847
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying, Lucene Directory
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
> Assignee: Sanne Grinovero
> Priority: Critical
>
> This JIRA is focused on optimizing performance of use case where the application uses short or none transactions - therefore, no batching can be applied - and then performs the query and expects that the result will be consistent (please, specify in documentation if any lag after cache.put() or tx.commit() is required and if the application can detect that the update has been applied).
> Performance of indexing is currently insufficient, when compared to competitors. Competitors show very low overhead of indexed writes, while we can see that the throughput is thousands of times lower [1] (configuration in [2]) with distributed mode storing index in replicated cache, and about 4x slowdown when comparing replicated cache without indexing and indexing to NRT FS/RAM.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
> [2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-3983) Remove some performance bottlenecks from ProtoStream
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-3983?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-3983:
---------------------------------------
Could we close this now and consider it resolved?
Let's classify further improvements in dedicated issues, and most importantly let's re-evaluate priorities over other tasks.
> Remove some performance bottlenecks from ProtoStream
> ----------------------------------------------------
>
> Key: ISPN-3983
> URL: https://issues.jboss.org/browse/ISPN-3983
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 6.0.0.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Labels: 6.3remotequeries, 630, 64QueryBlockers
> Fix For: 7.0.0.CR2, 7.0.0.Final
>
>
> Profiling has shown that there are many HashMaps created in MessageContext to help field lookup by name.
> Normally this is a static piece of information that should be computed only once when a marshaller is registered and then it should become immutable and be reused.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4847) Improve indexing performance
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4847?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-4847:
---------------------------------------
I'm going to link to some more HSEARCH issues, but let's not forget that these need an integration task too: a Search release and bump the dependency in Infinispan.
Rather than creating specific tasks for each, let's make sure that this issue isn't resolved until
- all dependant issues are integrated
- all Search dependendant issues are actually included (and enabled!) in the Infinispan branch
- and of course that performance is satisfactory
> Improve indexing performance
> ----------------------------
>
> Key: ISPN-4847
> URL: https://issues.jboss.org/browse/ISPN-4847
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying, Lucene Directory
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
> Assignee: Sanne Grinovero
> Priority: Critical
>
> This JIRA is focused on optimizing performance of use case where the application uses short or none transactions - therefore, no batching can be applied - and then performs the query and expects that the result will be consistent (please, specify in documentation if any lag after cache.put() or tx.commit() is required and if the application can detect that the update has been applied).
> Performance of indexing is currently insufficient, when compared to competitors. Competitors show very low overhead of indexed writes, while we can see that the throughput is thousands of times lower [1] (configuration in [2]) with distributed mode storing index in replicated cache, and about 4x slowdown when comparing replicated cache without indexing and indexing to NRT FS/RAM.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
> [2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4855) IndexManager start failed
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4855?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-4855:
---------------------------------------
Thanks Radim, but I think this might be ISPN-4807 ? You're not testing master are you?
> IndexManager start failed
> -------------------------
>
> Key: ISPN-4855
> URL: https://issues.jboss.org/browse/ISPN-4855
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
>
> I have experienced this exception that caused the cacheManager startup to fail:
> Note that this was in library mode, though hotrod client was on the classpath.
> {code}
> org.hibernate.search.exception.SearchException: HSEARCH000103: Unable to initialize IndexManager named 'org.infinispan.query.remote.indexing.ProtobufValueWrapper'
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:260)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:513)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:482)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:91)
> at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:369)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildNewSearchFactory(SearchFactoryBuilder.java:208)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:126)
> at org.infinispan.query.impl.LifecycleManager.getSearchFactory(LifecycleManager.java:261)
> at org.infinispan.query.impl.LifecycleManager.cacheStarting(LifecycleManager.java:103)
> at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:228)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:214)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:762)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
> at org.radargun.service.InfinispanEmbeddedService.startCaches(InfinispanEmbeddedService.java:111)
> at org.radargun.service.Infinispan51EmbeddedService.startCaches(Infinispan51EmbeddedService.java:95)
> at org.radargun.service.InfinispanLifecycle.start(InfinispanLifecycle.java:45)
> at org.radargun.service.InfinispanKillableLifecycle.start(InfinispanKillableLifecycle.java:47)
> at org.radargun.stages.lifecycle.LifecycleHelper.start(LifecycleHelper.java:57)
> at org.radargun.stages.lifecycle.ServiceStartStage.executeOnSlave(ServiceStartStage.java:76)
> at org.radargun.SlaveBase.scenarioLoop(SlaveBase.java:84)
> at org.radargun.SlaveBase$ScenarioRunner.run(SlaveBase.java:140)
> Caused by: java.lang.NullPointerException
> at org.infinispan.lucene.readlocks.DistributedSegmentReadLocker.deleteOrReleaseReadLock(DistributedSegmentReadLocker.java:73)
> at org.infinispan.lucene.impl.DirectoryImplementor.deleteFile(DirectoryImplementor.java:65)
> at org.infinispan.lucene.impl.DirectoryLuceneV4.deleteFile(DirectoryLuceneV4.java:64)
> at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:595)
> at org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:249)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:781)
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:146)
> at org.hibernate.search.infinispan.impl.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:73)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:90)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> ... 22 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4855) IndexManager start failed
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-4855?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-4855:
------------------------------
Component/s: Embedded Querying
> IndexManager start failed
> -------------------------
>
> Key: ISPN-4855
> URL: https://issues.jboss.org/browse/ISPN-4855
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
>
> I have experienced this exception that caused the cacheManager startup to fail:
> Note that this was in library mode, though hotrod client was on the classpath.
> {code}
> org.hibernate.search.exception.SearchException: HSEARCH000103: Unable to initialize IndexManager named 'org.infinispan.query.remote.indexing.ProtobufValueWrapper'
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:260)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:513)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:482)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:91)
> at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:369)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildNewSearchFactory(SearchFactoryBuilder.java:208)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:126)
> at org.infinispan.query.impl.LifecycleManager.getSearchFactory(LifecycleManager.java:261)
> at org.infinispan.query.impl.LifecycleManager.cacheStarting(LifecycleManager.java:103)
> at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:228)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:214)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:762)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
> at org.radargun.service.InfinispanEmbeddedService.startCaches(InfinispanEmbeddedService.java:111)
> at org.radargun.service.Infinispan51EmbeddedService.startCaches(Infinispan51EmbeddedService.java:95)
> at org.radargun.service.InfinispanLifecycle.start(InfinispanLifecycle.java:45)
> at org.radargun.service.InfinispanKillableLifecycle.start(InfinispanKillableLifecycle.java:47)
> at org.radargun.stages.lifecycle.LifecycleHelper.start(LifecycleHelper.java:57)
> at org.radargun.stages.lifecycle.ServiceStartStage.executeOnSlave(ServiceStartStage.java:76)
> at org.radargun.SlaveBase.scenarioLoop(SlaveBase.java:84)
> at org.radargun.SlaveBase$ScenarioRunner.run(SlaveBase.java:140)
> Caused by: java.lang.NullPointerException
> at org.infinispan.lucene.readlocks.DistributedSegmentReadLocker.deleteOrReleaseReadLock(DistributedSegmentReadLocker.java:73)
> at org.infinispan.lucene.impl.DirectoryImplementor.deleteFile(DirectoryImplementor.java:65)
> at org.infinispan.lucene.impl.DirectoryLuceneV4.deleteFile(DirectoryLuceneV4.java:64)
> at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:595)
> at org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:249)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:781)
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:146)
> at org.hibernate.search.infinispan.impl.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:73)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:90)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> ... 22 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4855) IndexManager start failed
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-4855?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-4855:
------------------------------
Affects Version/s: 7.0.0.CR1
> IndexManager start failed
> -------------------------
>
> Key: ISPN-4855
> URL: https://issues.jboss.org/browse/ISPN-4855
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
>
> I have experienced this exception that caused the cacheManager startup to fail:
> Note that this was in library mode, though hotrod client was on the classpath.
> {code}
> org.hibernate.search.exception.SearchException: HSEARCH000103: Unable to initialize IndexManager named 'org.infinispan.query.remote.indexing.ProtobufValueWrapper'
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:260)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:513)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:482)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:91)
> at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:369)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildNewSearchFactory(SearchFactoryBuilder.java:208)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:126)
> at org.infinispan.query.impl.LifecycleManager.getSearchFactory(LifecycleManager.java:261)
> at org.infinispan.query.impl.LifecycleManager.cacheStarting(LifecycleManager.java:103)
> at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:228)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:214)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:762)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
> at org.radargun.service.InfinispanEmbeddedService.startCaches(InfinispanEmbeddedService.java:111)
> at org.radargun.service.Infinispan51EmbeddedService.startCaches(Infinispan51EmbeddedService.java:95)
> at org.radargun.service.InfinispanLifecycle.start(InfinispanLifecycle.java:45)
> at org.radargun.service.InfinispanKillableLifecycle.start(InfinispanKillableLifecycle.java:47)
> at org.radargun.stages.lifecycle.LifecycleHelper.start(LifecycleHelper.java:57)
> at org.radargun.stages.lifecycle.ServiceStartStage.executeOnSlave(ServiceStartStage.java:76)
> at org.radargun.SlaveBase.scenarioLoop(SlaveBase.java:84)
> at org.radargun.SlaveBase$ScenarioRunner.run(SlaveBase.java:140)
> Caused by: java.lang.NullPointerException
> at org.infinispan.lucene.readlocks.DistributedSegmentReadLocker.deleteOrReleaseReadLock(DistributedSegmentReadLocker.java:73)
> at org.infinispan.lucene.impl.DirectoryImplementor.deleteFile(DirectoryImplementor.java:65)
> at org.infinispan.lucene.impl.DirectoryLuceneV4.deleteFile(DirectoryLuceneV4.java:64)
> at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:595)
> at org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:249)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:781)
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:146)
> at org.hibernate.search.infinispan.impl.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:73)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:90)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> ... 22 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months