[JBoss JIRA] (ISPN-951) Infinispan Query module should load entities using getAsync
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-951?page=com.atlassian.jira.plugin.s... ]
Sanne Grinovero updated ISPN-951:
---------------------------------
Description:
Ideally we should use a method like {{List getAll(Object[] keys)}} from Infinispan core, but that doesn't exist yet. Such a method would be able to split the keys in such a way to minimize the number of RPCs needed to "cover" the set of all segments needed to satisfy the query, so to find the minimum set of nodes available which need to be contacted to retrieve all entries.
Loading of each entry doesn't necessarily need to happen sequentially, I guess the minimum set of RPCs should be executed in parallel, then we block until they are all returned and need to restore the order of the list (the order of the return elements need to match the order of their respective keys from the parameter).
We should implement such a method, or until that's possible it might be easier to run each individual get in a {{getAsync}} so that the latency of a Query returning for example 100 entries doesn't scale up to 100 sequential RPCs.
Requires migration to new Loader API from Hibernate Search.
was:Requires migration to new Loader API from Hibernate Search
> Infinispan Query module should load entities using getAsync
> -----------------------------------------------------------
>
> Key: ISPN-951
> URL: https://issues.jboss.org/browse/ISPN-951
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
>
> Ideally we should use a method like {{List getAll(Object[] keys)}} from Infinispan core, but that doesn't exist yet. Such a method would be able to split the keys in such a way to minimize the number of RPCs needed to "cover" the set of all segments needed to satisfy the query, so to find the minimum set of nodes available which need to be contacted to retrieve all entries.
> Loading of each entry doesn't necessarily need to happen sequentially, I guess the minimum set of RPCs should be executed in parallel, then we block until they are all returned and need to restore the order of the list (the order of the return elements need to match the order of their respective keys from the parameter).
> We should implement such a method, or until that's possible it might be easier to run each individual get in a {{getAsync}} so that the latency of a Query returning for example 100 entries doesn't scale up to 100 sequential RPCs.
> Requires migration to new Loader API from Hibernate Search.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4847) Improve indexing performance
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4847?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-4847:
----------------------------------
Assignee: Gustavo Fernandes (was: Sanne Grinovero)
> Improve indexing performance
> ----------------------------
>
> Key: ISPN-4847
> URL: https://issues.jboss.org/browse/ISPN-4847
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying, Lucene Directory
> Affects Versions: 7.0.0.CR1
> Reporter: Radim Vansa
> Assignee: Gustavo Fernandes
> Priority: Critical
>
> This JIRA is focused on optimizing performance of use case where the application uses short or none transactions - therefore, no batching can be applied - and then performs the query and expects that the result will be consistent (please, specify in documentation if any lag after cache.put() or tx.commit() is required and if the application can detect that the update has been applied).
> Performance of indexing is currently insufficient, when compared to competitors. Competitors show very low overhead of indexed writes, while we can see that the throughput is thousands of times lower [1] (configuration in [2]) with distributed mode storing index in replicated cache, and about 4x slowdown when comparing replicated cache without indexing and indexing to NRT FS/RAM.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
> [2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-index...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-2842) Failure in data insertion into 4 node topology aware cluster with DIST_SYNC mode & enabled InfinispanIndexManager
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-2842?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero updated ISPN-2842:
----------------------------------
Assignee: Gustavo Fernandes (was: Sanne Grinovero)
> Failure in data insertion into 4 node topology aware cluster with DIST_SYNC mode & enabled InfinispanIndexManager
> ------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-2842
> URL: https://issues.jboss.org/browse/ISPN-2842
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 5.2.1.Final
> Reporter: Anna Manukyan
> Assignee: Gustavo Fernandes
> Labels: 64QueryBlockers, stable_embedded_query
>
> We have tests which run queries on TopologyAware distributed cache nodes (2 nodes) with enabled indexing and infinispan directory_provider.
> I decided to see what will happen if we will start 4 such nodes, and found out that in this case when trying during the data put into the cache on the first node, the following exception appears:
> {code}
> 2013-02-21 14:27:55,958 ERROR [InvocationContextInterceptor] (OOB-2,ISPN,NodeD-4638(test3)) ISPN000136: Execution error
> org.hibernate.search.SearchException: HSEARCH000103: Unable to initialize IndexManager person
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:230)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:102)
> at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:414)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildIncrementalSearchFactory(SearchFactoryBuilder.java:169)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:149)
> at org.hibernate.search.impl.MutableSearchFactory.addClasses(MutableSearchFactory.java:194)
> at org.infinispan.query.backend.QueryInterceptor.enableClassesIncrementally(QueryInterceptor.java:225)
> at org.infinispan.query.backend.QueryInterceptor.updateKnownTypesIfNeeded(QueryInterceptor.java:250)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:426)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:128)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitPutKeyValueCommand(NonTransactionalLockingInterceptor.java:84)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:61)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:70)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:101)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:122)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:86)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:247)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:220)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:472)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:664)
> at org.jgroups.JChannel.up(JChannel.java:719)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1007)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:183)
> at org.jgroups.protocols.FC.up(FC.java:479)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
> at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:787)
> at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:411)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:609)
> at org.jgroups.protocols.Discovery.up(Discovery.java:371)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1267)
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1398)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.hibernate.search.SearchException: Unable to open Lucene IndexReader for IndexManager person
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:125)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.initialize(SharingBufferReaderProvider.java:103)
> at org.hibernate.search.indexes.impl.CommonPropertiesParse.createDirectoryBasedReaderProvider(CommonPropertiesParse.java:164)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.createIndexReader(DirectoryBasedIndexManager.java:228)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:105)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:227)
> ... 58 more
> Caused by: java.io.IOException: Read past EOF
> at org.infinispan.lucene.impl.SingleChunkIndexInput.readByte(SingleChunkIndexInput.java:76)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107)
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:314)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:74)
> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:80)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
> at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
> at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:143)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:217)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:120)
> ... 63 more
> 2013-02-21 14:27:55,962 ERROR [InboundInvocationHandlerImpl] (OOB-2,ISPN,NodeD-4638(test3)) Exception executing command
> org.hibernate.search.SearchException: HSEARCH000103: Unable to initialize IndexManager person
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:230)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:102)
> at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:414)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildIncrementalSearchFactory(SearchFactoryBuilder.java:169)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:149)
> at org.hibernate.search.impl.MutableSearchFactory.addClasses(MutableSearchFactory.java:194)
> at org.infinispan.query.backend.QueryInterceptor.enableClassesIncrementally(QueryInterceptor.java:225)
> at org.infinispan.query.backend.QueryInterceptor.updateKnownTypesIfNeeded(QueryInterceptor.java:250)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:426)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:128)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitPutKeyValueCommand(NonTransactionalLockingInterceptor.java:84)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:61)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:70)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:101)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:122)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:86)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:247)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:220)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:472)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:664)
> at org.jgroups.JChannel.up(JChannel.java:719)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1007)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:183)
> at org.jgroups.protocols.FC.up(FC.java:479)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
> at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:787)
> at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:411)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:609)
> at org.jgroups.protocols.Discovery.up(Discovery.java:371)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1267)
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1398)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.hibernate.search.SearchException: Unable to open Lucene IndexReader for IndexManager person
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:125)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.initialize(SharingBufferReaderProvider.java:103)
> at org.hibernate.search.indexes.impl.CommonPropertiesParse.createDirectoryBasedReaderProvider(CommonPropertiesParse.java:164)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.createIndexReader(DirectoryBasedIndexManager.java:228)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:105)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:227)
> ... 58 more
> Caused by: java.io.IOException: Read past EOF
> at org.infinispan.lucene.impl.SingleChunkIndexInput.readByte(SingleChunkIndexInput.java:76)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107)
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:314)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:74)
> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:80)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
> at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
> at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:143)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:217)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:120)
> ... 63 more
> 2013-02-21 14:27:55,981 ERROR [InvocationContextInterceptor] (main) ISPN000136: Execution error
> org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeD-4638(test3), see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:72)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:310)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:179)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:515)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:175)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:197)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:254)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:241)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:236)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:231)
> at org.infinispan.interceptors.distribution.NonTxConcurrentDistributionInterceptor.handleLocalWrite(NonTxConcurrentDistributionInterceptor.java:104)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleWriteCommand(NonTxDistributionInterceptor.java:94)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:68)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:275)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:166)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:127)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitPutKeyValueCommand(NonTransactionalLockingInterceptor.java:82)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1162)
> at org.infinispan.CacheImpl.putInternal(CacheImpl.java:760)
> at org.infinispan.CacheImpl.put(CacheImpl.java:754)
> at org.infinispan.CacheImpl.put(CacheImpl.java:748)
> at org.infinispan.CacheSupport.put(CacheSupport.java:53)
> at org.infinispan.query.blackbox.TopologyAware4NodeClusteredCacheTest.prepareTestData(TopologyAware4NodeClusteredCacheTest.java:101)
> at org.infinispan.query.blackbox.TopologyAware4NodeClusteredCacheTest.testSimple(TopologyAware4NodeClusteredCacheTest.java:107)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
> at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
> at org.testng.SuiteRunner.run(SuiteRunner.java:240)
> at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
> at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
> at org.testng.TestNG.runSuitesSequentially(TestNG.java:1198)
> at org.testng.TestNG.runSuitesLocally(TestNG.java:1123)
> at org.testng.TestNG.run(TestNG.java:1031)
> at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
> at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
> at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
> at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:111)
> Caused by: org.hibernate.search.SearchException: HSEARCH000103: Unable to initialize IndexManager person
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:230)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:102)
> at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:414)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildIncrementalSearchFactory(SearchFactoryBuilder.java:169)
> at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:149)
> at org.hibernate.search.impl.MutableSearchFactory.addClasses(MutableSearchFactory.java:194)
> at org.infinispan.query.backend.QueryInterceptor.enableClassesIncrementally(QueryInterceptor.java:225)
> at org.infinispan.query.backend.QueryInterceptor.updateKnownTypesIfNeeded(QueryInterceptor.java:250)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:426)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:128)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitPutKeyValueCommand(NonTransactionalLockingInterceptor.java:84)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:61)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:70)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:101)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:122)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:86)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:247)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:220)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:472)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:664)
> at org.jgroups.JChannel.up(JChannel.java:719)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1007)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:183)
> at org.jgroups.protocols.FC.up(FC.java:479)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
> at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:787)
> at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:411)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:609)
> at org.jgroups.protocols.Discovery.up(Discovery.java:371)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1267)
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1398)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.hibernate.search.SearchException: Unable to open Lucene IndexReader for IndexManager person
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:125)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.initialize(SharingBufferReaderProvider.java:103)
> at org.hibernate.search.indexes.impl.CommonPropertiesParse.createDirectoryBasedReaderProvider(CommonPropertiesParse.java:164)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.createIndexReader(DirectoryBasedIndexManager.java:228)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:105)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:227)
> ... 58 more
> Caused by: java.io.IOException: Read past EOF
> at org.infinispan.lucene.impl.SingleChunkIndexInput.readByte(SingleChunkIndexInput.java:76)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107)
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:314)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:74)
> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:80)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
> at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
> at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:143)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:217)
> at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:120)
> ... 63 more
> {code}
> Please note, that the exception appears after the first put. So the code is the following:
> {code}
> List caches = new ArrayList();
> ConfigurationBuilder builder = AbstractCacheTest.getDefaultClusteredCacheConfig(CacheMode.DIST_SYNC, false);
> builder.indexing().enable().indexLocalOnly(true);
> builder.indexing().addProperty("hibernate.search.default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager")
> .addProperty("default.directory_provider", "infinispan")
> .addProperty("hibernate.search.default.exclusive_index_use", "false")
> .addProperty("lucene_version", "LUCENE_36");
> for(int i = 0; i < 4; i++) {
> GlobalConfigurationBuilder globalConfigurationBuilder = GlobalConfigurationBuilder
> .defaultClusteredBuilder();
> globalConfigurationBuilder.transport().machineId("a" + i).rackId("b" + i).siteId("test" + i);
> EmbeddedCacheManager cm1 = TestCacheManagerFactory.createClusteredCacheManager(
> globalConfigurationBuilder, builder);
> caches.add(cm1.getCache());
> }
> for(Object cache : caches) {
> cacheManagers.add(((Cache) cache).getCacheManager());
> }
> waitForClusterToForm();
> cache1 = (Cache<String, Person>) caches.get(0);
> cache2 = (Cache<String, Person>) caches.get(1);
> cache3 = (Cache<String, Person>) caches.get(2);
> Person person1 = initialize(); //person1 initialization
> Person person2 = initialize(); //person1 initialization
> cache1.put(key1, person1);
> cache1.put(key2, person2); //<----- The failure takes place on this line
> {code}
> I also tried to increase the numOwners from 2 to 4, but it doesn't help. This issue appears when creating the NON_TRANSACTIONAL cache.
> I tried to create the same configuration but with TRANSACTION enabled, in this case the issue ISPN-2815 appears.
> Also, I have tried the same configuration but with REPL_SYNC mode, and the execution passed.
> Another note: in case of 3 nodes -> 2 num owners, the test passes, but starting from 4 nodes - this exception appears.
> I've tried to set the number of nodes to 2 + I've set the value for numOwners to 1: again the same failure appears.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4848) Offer way to implement includeCurrentState based on indexed query
by Emmanuel Bernard (JIRA)
Emmanuel Bernard created ISPN-4848:
--------------------------------------
Summary: Offer way to implement includeCurrentState based on indexed query
Key: ISPN-4848
URL: https://issues.jboss.org/browse/ISPN-4848
Project: Infinispan
Issue Type: Feature Request
Reporter: Emmanuel Bernard
Assignee: Mircea Markus
Based on the infinispan-dev mailing list discussion from September 2014 titled
'Feedback and requests on clustered and remote listeners'.
Loading the whole state of the data grid to then filter / convert them can be costly especially if data has been passivated.
An alternative could be offered that would delegate the includeCurrentState filtering with a global indexed query. That query would need to be provided by the user. While the query is run, the list of change events matching should be piled up and released once the query has be run and the converted events are send to the clustered listener.
Maybe that should be only done for "continuous" queries as we could use the query for both the initial and continuous side of the query transparently.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4566) ManualIndexingTest.testManualIndexing random failures
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4566?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero updated ISPN-4566:
----------------------------------
Assignee: (was: Sanne Grinovero)
> ManualIndexingTest.testManualIndexing random failures
> -----------------------------------------------------
>
> Key: ISPN-4566
> URL: https://issues.jboss.org/browse/ISPN-4566
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Query
> Affects Versions: 7.0.0.Alpha5
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 7.0.0.CR2
>
>
> Random timeouts when TRACE logging is enabled:
> {noformat}
> 04:58:33,679 ERROR (testng-ManualIndexingTest:) [UnitTestTestNGListener] Test testManualIndexing(org.infinispan.query.api.ManualIndexingTest) failed.
> org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: Map phase executing at ManualIndexingTest-NodeA-44176 did not complete within 20 sec timeout
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeHelper(MapReduceTask.java:506)
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:407)
> at org.infinispan.query.impl.massindex.MapReduceMassIndexer.start(MapReduceMassIndexer.java:25)
> at org.infinispan.query.api.ManualIndexingTest.testManualIndexing(ManualIndexingTest.java:52)
> {noformat}
> Trace log here: http://ci.infinispan.org/viewLog.html?buildId=9816&buildTypeId=Infinispan...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months