[JBoss JIRA] (ISPN-4702) The ForkJoin thread pool should be started lazily
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4702?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero reassigned ISPN-4702:
-------------------------------------
Assignee: Sanne Grinovero (was: Dan Berindei)
> The ForkJoin thread pool should be started lazily
> -------------------------------------------------
>
> Key: ISPN-4702
> URL: https://issues.jboss.org/browse/ISPN-4702
> Project: Infinispan
> Issue Type: Enhancement
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 7.0.0.Beta1
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Priority: Minor
>
> I'm not using these features, yet a significant amount of threads are being started.
> We should at least start this pool only on-demand, and ideally shut it down after a grace period if it's no longer being used.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4702) The ForkJoin thread pool should be started lazily
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4702?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-4702:
---------------------------------------
Ok I might be wrong then, I thought this would be started only in case of Map/Reduce jobs, which I wasn't doing so these threads which I was seeing in the stacktraces seemed they shouldn't have been there.
But I didn't know it was used by {{EquivalentConcurrentHashMapV8}}, that might explain it.. I'll need to check my test again, so I'll take this issue for me to gather a better description or eventually reject it.
Thanks!
> The ForkJoin thread pool should be started lazily
> -------------------------------------------------
>
> Key: ISPN-4702
> URL: https://issues.jboss.org/browse/ISPN-4702
> Project: Infinispan
> Issue Type: Enhancement
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 7.0.0.Beta1
> Reporter: Sanne Grinovero
> Assignee: Dan Berindei
> Priority: Minor
>
> I'm not using these features, yet a significant amount of threads are being started.
> We should at least start this pool only on-demand, and ideally shut it down after a grace period if it's no longer being used.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4595) InfinispanIndexManager fails to pick indexing Node on preload
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-4595?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes reassigned ISPN-4595:
---------------------------------------
Assignee: Gustavo Fernandes
> InfinispanIndexManager fails to pick indexing Node on preload
> -------------------------------------------------------------
>
> Key: ISPN-4595
> URL: https://issues.jboss.org/browse/ISPN-4595
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 7.0.0.Alpha5
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Critical
>
> Configuration:
> {code:java}
> GlobalConfigurationBuilder globalConfigurationBuilder = new GlobalConfigurationBuilder().clusteredDefault();
> return new DefaultCacheManager(
> globalConfigurationBuilder
> .globalJmxStatistics().enable()
> .build(),
> new ConfigurationBuilder()
> .clustering().cacheMode(CacheMode.REPL_SYNC)
> .jmxStatistics().enable()
> .persistence()
> .addSingleFileStore()
> .location("/usr/local/store")
> .preload(true).shared(true)
> .indexing().index(Index.LOCAL)
> .addProperty("default.directory_provider", "infinispan")
> .addProperty("default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager")
> .build()
> );
> }
> {code}
> Exception:
> {code}
> ERROR: ISPN000136: Execution error
> java.lang.NullPointerException
> at org.infinispan.distribution.impl.DistributionManagerImpl.getWriteConsistentHash(DistributionManagerImpl.java:115)
> at org.infinispan.distribution.impl.DistributionManagerImpl.getConsistentHash(DistributionManagerImpl.java:105)
> at org.infinispan.distribution.impl.DistributionManagerImpl.getPrimaryLocation(DistributionManagerImpl.java:95)
> at org.infinispan.query.indexmanager.InfinispanCommandsBackend.getPrimaryNodeAddress(InfinispanCommandsBackend.java:135)
> at org.infinispan.query.indexmanager.InfinispanCommandsBackend.isMasterLocal(InfinispanCommandsBackend.java:113)
> at org.infinispan.query.indexmanager.MasterSwitchDelegatingQueueProcessor.applyWork(MasterSwitchDelegatingQueueProcessor.java:47)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.performOperations(DirectoryBasedIndexManager.java:113)
> at org.hibernate.search.backend.impl.WorkQueuePerIndexSplitter.commitOperations(WorkQueuePerIndexSplitter.java:49)
> at org.hibernate.search.backend.impl.BatchedQueueingProcessor.performWorks(BatchedQueueingProcessor.java:82)
> at org.hibernate.search.backend.impl.TransactionalWorker.performWork(TransactionalWorker.java:86)
> at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:232)
> at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:226)
> at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:220)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:522)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:161)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:46)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:159)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:95)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:111)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:74)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1482)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:960)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:952)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:483)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.preloadKey(PersistenceManagerImpl.java:641)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.access$000(PersistenceManagerImpl.java:69)
> at org.infinispan.persistence.manager.PersistenceManagerImpl$1.processEntry(PersistenceManagerImpl.java:227)
> at org.infinispan.persistence.file.SingleFileStore$1.call(SingleFileStore.java:529)
> at org.infinispan.persistence.file.SingleFileStore$1.call(SingleFileStore.java:523)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)
> at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
> at org.infinispan.executors.ExecutorAllCompletionService.submit(ExecutorAllCompletionService.java:31)
> at org.infinispan.persistence.file.SingleFileStore.process(SingleFileStore.java:523)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.preload(PersistenceManagerImpl.java:219)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:752)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:574)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:529)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:409)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4822) MassIndexer is not ClusteredQuery friendly
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-4822?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-4822:
------------------------------------
Status: Open (was: New)
> MassIndexer is not ClusteredQuery friendly
> ------------------------------------------
>
> Key: ISPN-4822
> URL: https://issues.jboss.org/browse/ISPN-4822
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 7.0.0.CR1
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
>
> The MassIndexer assumes a cluster-wide shared index, and thus calls
> wipeExistingIndexes() only once in the node where it is invoked, the same apply for flush operation after the all mappers and reducers run.
> When using Clustered Queries, indexes are locally scoped and independent , and need to be wiped and flushed on every node.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4823) ClusteredQueryMassIndexingTest always fails
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-4823?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-4823:
------------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2948
> ClusteredQueryMassIndexingTest always fails
> -------------------------------------------
>
> Key: ISPN-4823
> URL: https://issues.jboss.org/browse/ISPN-4823
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Query
> Affects Versions: 7.0.0.CR1
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
>
> Test is wrongly marked as unstable but it fails every time.
> {code}
> Transport protocol stack used = tcp
> java.lang.AssertionError:
> Expected :4
> Actual :1
> <Click to see difference>
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertEquals(Assert.java:123)
> at org.testng.Assert.assertEquals(Assert.java:370)
> at org.testng.Assert.assertEquals(Assert.java:380)
> at org.infinispan.query.distributed.ClusteredQueryMassIndexingTest.verifyFindsCar(ClusteredQueryMassIndexingTest.java:30)
> at org.infinispan.query.distributed.DistributedMassIndexingTest.verifyFindsCar(DistributedMassIndexingTest.java:82)
> at org.infinispan.query.distributed.DistributedMassIndexingTest.testReindexing(DistributedMassIndexingTest.java:51)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4847) Improve indexing performance
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4847?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero reassigned ISPN-4847:
-------------------------------------
Assignee: Sanne Grinovero (was: Gustavo Fernandes)
> Improve indexing performance
> ----------------------------
>
> Key: ISPN-4847
> URL: https://issues.jboss.org/browse/ISPN-4847
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying, Lucene Directory
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
> Assignee: Sanne Grinovero
> Priority: Critical
>
> This JIRA is focused on optimizing performance of use case where the application uses short or none transactions - therefore, no batching can be applied - and then performs the query and expects that the result will be consistent (please, specify in documentation if any lag after cache.put() or tx.commit() is required and if the application can detect that the update has been applied).
> Performance of indexing is currently insufficient, when compared to competitors. Competitors show very low overhead of indexed writes, while we can see that the throughput is thousands of times lower [1] (configuration in [2]) with distributed mode storing index in replicated cache, and about 4x slowdown when comparing replicated cache without indexing and indexing to NRT FS/RAM.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
> [2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4849) Offer ability to read the value in raw marshalled form in clustered listener filter and converter
by Emmanuel Bernard (JIRA)
Emmanuel Bernard created ISPN-4849:
--------------------------------------
Summary: Offer ability to read the value in raw marshalled form in clustered listener filter and converter
Key: ISPN-4849
URL: https://issues.jboss.org/browse/ISPN-4849
Project: Infinispan
Issue Type: Feature Request
Reporter: Emmanuel Bernard
Assignee: Mircea Markus
To avoid the need to ship the value class to all the data grid classpath, it would be quite useful to ask to read the value as byte[] / marshalled form for the filter and converter part on each node involved (for a clustered listener).
For example, if the value is serialized as protobuf, an implementation could receive the byte[] of the protobuf and read the necessary properties without forcing a specific application .class file to be present. It also saves in unnecessary unmarshalling. Ideally a flag could be used @Listener(marshalled=false).
This issue might have been created (by Galder) but I cannot find it.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-951) Infinispan Query module should load entities using getAsync
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-951?page=com.atlassian.jira.plugin.s... ]
Sanne Grinovero updated ISPN-951:
---------------------------------
Assignee: Gustavo Fernandes (was: Sanne Grinovero)
> Infinispan Query module should load entities using getAsync
> -----------------------------------------------------------
>
> Key: ISPN-951
> URL: https://issues.jboss.org/browse/ISPN-951
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Gustavo Fernandes
>
> Ideally we should use a method like {{List getAll(Object[] keys)}} from Infinispan core, but that doesn't exist yet. Such a method would be able to split the keys in such a way to minimize the number of RPCs needed to "cover" the set of all segments needed to satisfy the query, so to find the minimum set of nodes available which need to be contacted to retrieve all entries.
> Loading of each entry doesn't necessarily need to happen sequentially, I guess the minimum set of RPCs should be executed in parallel, then we block until they are all returned and need to restore the order of the list (the order of the return elements need to match the order of their respective keys from the parameter).
> We should implement such a method, or until that's possible it might be easier to run each individual get in a {{getAsync}} so that the latency of a Query returning for example 100 entries doesn't scale up to 100 sequential RPCs.
> Requires migration to new Loader API from Hibernate Search.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months