[JBoss JIRA] (ISPN-6366) NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
by Mehdi Rakhshani (JIRA)
[ https://issues.jboss.org/browse/ISPN-6366?page=com.atlassian.jira.plugin.... ]
Mehdi Rakhshani updated ISPN-6366:
----------------------------------
Attachment: jbossDataGridConfiguration.xml
Infinispan configuration file of cache managers.
> NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6366
> URL: https://issues.jboss.org/browse/ISPN-6366
> Project: Infinispan
> Issue Type: Bug
> Environment: RH LINUX
> Reporter: Mehdi Rakhshani
> Attachments: jbossDataGridConfiguration.xml
>
>
> We have 4 instances of our application that uses Infinispan in a replicated, synchronized cluster. There are multiple independent clusters running in different data centers which share the same DB so the content of the caches is the same everywhere. The instances have been running for weeks with very little activity (the internal client application that comes in via REST API to access these apps has not been brought on live yet).
> Of the 4 cluster members, 2 (app servers) use the cache for read-only, 2 (cache managers) keep the cache up to date from the DB. The app servers keep 20,000 records in the cache and the rest in a file store. The cache managers keep all records in the cache and do not use a file store. All cluster members are configured to get a copy of the cache from others in the cluster during startup.
> Without any changes, when the two tomcat containers hosting the cache mangers were recycled, they stopped coming up. The following exception is thrown in each cache manager in catalina.out.
> The key for this cache ('EdiCache') is a 39 byte string. It is made up of two left-padded strings concatenated together. The key in the exception contains '+' which should be blank instead. When I dump the caches from a cluster that is running in a different data center, I see that the key is '128738 ' not '128738+++++++++++++++++++++++++++++++++'.
> We are using version: 8.1.0.Final
> I can upload catalina.out, my application logs and Infinispan configuration files if needed.
> Any idea what is the cause of this error?
> 2016-03-09 07:29:02,106 [remote-thread--p2-t4] ERROR org.infinispan.interceptors.InvocationContextInterceptor - ISPN000136: Error executing command PutKeyValueCommand, writing keys [128738+++++++++++++++++++++++++++++++++]
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-03-09 07:29:02,111 [remote-thread--p2-t4] WARN org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler - ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='EdiCache', command=PutKeyValueCommand{key=128738+++++++++++++++++++++++++++++++++, value=ImBusinessUnitRootEdi [address=128738, qualifier=null, buId=GC00029665M1], flags=[IGNORE_RETURN_VALUES], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true}}
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (ISPN-6349) Add primitive CacheStream interfaces
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-6349?page=com.atlassian.jira.plugin.... ]
William Burns updated ISPN-6349:
--------------------------------
Status: Open (was: New)
> Add primitive CacheStream interfaces
> ------------------------------------
>
> Key: ISPN-6349
> URL: https://issues.jboss.org/browse/ISPN-6349
> Project: Infinispan
> Issue Type: Enhancement
> Components: Distributed Execution and Map/Reduce
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.0.0.Alpha1
>
>
> With ISPN-6272 we now have full CacheStream support for returns on operator invocations. e should also do this for the primitive interfaces (DoubleStream/IntStream/LongStream). The thought is to call them CacheDoubleStream/CacheIntStream/CacheLongStream.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (ISPN-6366) NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
by Mehdi Rakhshani (JIRA)
[ https://issues.jboss.org/browse/ISPN-6366?page=com.atlassian.jira.plugin.... ]
Mehdi Rakhshani commented on ISPN-6366:
---------------------------------------
There was no activity other than tomcat being restarted. My application does not appear in the call stack trace at all.
I am attaching the Infinispan configuration file. It shows that when a cluster member is started, its caches are populated at startup from other cluster members.
The keys originate from the DB. They are two columns in the DB (35 and 4), which are right-padded with blanks (sorry I said left padded before) and concatenated together. I am not sure where the plus signs are coming from. In the cluster running in another data center, when I call the application's REST API with the data (128738), it finds the record in the cache just fine. The app pads the data with blanks and searches the cache for it.
> NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6366
> URL: https://issues.jboss.org/browse/ISPN-6366
> Project: Infinispan
> Issue Type: Bug
> Environment: RH LINUX
> Reporter: Mehdi Rakhshani
>
> We have 4 instances of our application that uses Infinispan in a replicated, synchronized cluster. There are multiple independent clusters running in different data centers which share the same DB so the content of the caches is the same everywhere. The instances have been running for weeks with very little activity (the internal client application that comes in via REST API to access these apps has not been brought on live yet).
> Of the 4 cluster members, 2 (app servers) use the cache for read-only, 2 (cache managers) keep the cache up to date from the DB. The app servers keep 20,000 records in the cache and the rest in a file store. The cache managers keep all records in the cache and do not use a file store. All cluster members are configured to get a copy of the cache from others in the cluster during startup.
> Without any changes, when the two tomcat containers hosting the cache mangers were recycled, they stopped coming up. The following exception is thrown in each cache manager in catalina.out.
> The key for this cache ('EdiCache') is a 39 byte string. It is made up of two left-padded strings concatenated together. The key in the exception contains '+' which should be blank instead. When I dump the caches from a cluster that is running in a different data center, I see that the key is '128738 ' not '128738+++++++++++++++++++++++++++++++++'.
> We are using version: 8.1.0.Final
> I can upload catalina.out, my application logs and Infinispan configuration files if needed.
> Any idea what is the cause of this error?
> 2016-03-09 07:29:02,106 [remote-thread--p2-t4] ERROR org.infinispan.interceptors.InvocationContextInterceptor - ISPN000136: Error executing command PutKeyValueCommand, writing keys [128738+++++++++++++++++++++++++++++++++]
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-03-09 07:29:02,111 [remote-thread--p2-t4] WARN org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler - ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='EdiCache', command=PutKeyValueCommand{key=128738+++++++++++++++++++++++++++++++++, value=ImBusinessUnitRootEdi [address=128738, qualifier=null, buId=GC00029665M1], flags=[IGNORE_RETURN_VALUES], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true}}
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (ISPN-6366) NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-6366?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-6366:
-------------------------------------
Mehdi,
Can you describe what was going on at this point? You mention {quote} All cluster members are configured to get a copy of the cache from others in the cluster during startup.{quote} what does this entail?
Also what is the key you are storing here? Is the toString just broken or was the key changed in some way?
> NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6366
> URL: https://issues.jboss.org/browse/ISPN-6366
> Project: Infinispan
> Issue Type: Bug
> Environment: RH LINUX
> Reporter: Mehdi Rakhshani
>
> We have 4 instances of our application that uses Infinispan in a replicated, synchronized cluster. There are multiple independent clusters running in different data centers which share the same DB so the content of the caches is the same everywhere. The instances have been running for weeks with very little activity (the internal client application that comes in via REST API to access these apps has not been brought on live yet).
> Of the 4 cluster members, 2 (app servers) use the cache for read-only, 2 (cache managers) keep the cache up to date from the DB. The app servers keep 20,000 records in the cache and the rest in a file store. The cache managers keep all records in the cache and do not use a file store. All cluster members are configured to get a copy of the cache from others in the cluster during startup.
> Without any changes, when the two tomcat containers hosting the cache mangers were recycled, they stopped coming up. The following exception is thrown in each cache manager in catalina.out.
> The key for this cache ('EdiCache') is a 39 byte string. It is made up of two left-padded strings concatenated together. The key in the exception contains '+' which should be blank instead. When I dump the caches from a cluster that is running in a different data center, I see that the key is '128738 ' not '128738+++++++++++++++++++++++++++++++++'.
> We are using version: 8.1.0.Final
> I can upload catalina.out, my application logs and Infinispan configuration files if needed.
> Any idea what is the cause of this error?
> 2016-03-09 07:29:02,106 [remote-thread--p2-t4] ERROR org.infinispan.interceptors.InvocationContextInterceptor - ISPN000136: Error executing command PutKeyValueCommand, writing keys [128738+++++++++++++++++++++++++++++++++]
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-03-09 07:29:02,111 [remote-thread--p2-t4] WARN org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler - ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='EdiCache', command=PutKeyValueCommand{key=128738+++++++++++++++++++++++++++++++++, value=ImBusinessUnitRootEdi [address=128738, qualifier=null, buId=GC00029665M1], flags=[IGNORE_RETURN_VALUES], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true}}
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (ISPN-6366) NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-6366?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-6366:
-------------------------------------
I moved this over to Infinispan as it was mentioned this was for Infinispan 8.1.
> NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6366
> URL: https://issues.jboss.org/browse/ISPN-6366
> Project: Infinispan
> Issue Type: Bug
> Environment: RH LINUX
> Reporter: Mehdi Rakhshani
>
> We have 4 instances of our application that uses Infinispan in a replicated, synchronized cluster. There are multiple independent clusters running in different data centers which share the same DB so the content of the caches is the same everywhere. The instances have been running for weeks with very little activity (the internal client application that comes in via REST API to access these apps has not been brought on live yet).
> Of the 4 cluster members, 2 (app servers) use the cache for read-only, 2 (cache managers) keep the cache up to date from the DB. The app servers keep 20,000 records in the cache and the rest in a file store. The cache managers keep all records in the cache and do not use a file store. All cluster members are configured to get a copy of the cache from others in the cluster during startup.
> Without any changes, when the two tomcat containers hosting the cache mangers were recycled, they stopped coming up. The following exception is thrown in each cache manager in catalina.out.
> The key for this cache ('EdiCache') is a 39 byte string. It is made up of two left-padded strings concatenated together. The key in the exception contains '+' which should be blank instead. When I dump the caches from a cluster that is running in a different data center, I see that the key is '128738 ' not '128738+++++++++++++++++++++++++++++++++'.
> We are using version: 8.1.0.Final
> I can upload catalina.out, my application logs and Infinispan configuration files if needed.
> Any idea what is the cause of this error?
> 2016-03-09 07:29:02,106 [remote-thread--p2-t4] ERROR org.infinispan.interceptors.InvocationContextInterceptor - ISPN000136: Error executing command PutKeyValueCommand, writing keys [128738+++++++++++++++++++++++++++++++++]
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-03-09 07:29:02,111 [remote-thread--p2-t4] WARN org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler - ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='EdiCache', command=PutKeyValueCommand{key=128738+++++++++++++++++++++++++++++++++, value=ImBusinessUnitRootEdi [address=128738, qualifier=null, buId=GC00029665M1], flags=[IGNORE_RETURN_VALUES], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true}}
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (ISPN-6366) NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-6366?page=com.atlassian.jira.plugin.... ]
William Burns moved JDG-69 to ISPN-6366:
----------------------------------------
Project: Infinispan (was: JBoss Data Grid)
Key: ISPN-6366 (was: JDG-69)
Workflow: GIT Pull Request with Triage workflow (was: CDW with loose statuses v1)
> NPE during startup/initialization of cache from other cluster members in a distributed, synchronized cluster
> ------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6366
> URL: https://issues.jboss.org/browse/ISPN-6366
> Project: Infinispan
> Issue Type: Bug
> Environment: RH LINUX
> Reporter: Mehdi Rakhshani
>
> We have 4 instances of our application that uses Infinispan in a replicated, synchronized cluster. There are multiple independent clusters running in different data centers which share the same DB so the content of the caches is the same everywhere. The instances have been running for weeks with very little activity (the internal client application that comes in via REST API to access these apps has not been brought on live yet).
> Of the 4 cluster members, 2 (app servers) use the cache for read-only, 2 (cache managers) keep the cache up to date from the DB. The app servers keep 20,000 records in the cache and the rest in a file store. The cache managers keep all records in the cache and do not use a file store. All cluster members are configured to get a copy of the cache from others in the cluster during startup.
> Without any changes, when the two tomcat containers hosting the cache mangers were recycled, they stopped coming up. The following exception is thrown in each cache manager in catalina.out.
> The key for this cache ('EdiCache') is a 39 byte string. It is made up of two left-padded strings concatenated together. The key in the exception contains '+' which should be blank instead. When I dump the caches from a cluster that is running in a different data center, I see that the key is '128738 ' not '128738+++++++++++++++++++++++++++++++++'.
> We are using version: 8.1.0.Final
> I can upload catalina.out, my application logs and Infinispan configuration files if needed.
> Any idea what is the cause of this error?
> 2016-03-09 07:29:02,106 [remote-thread--p2-t4] ERROR org.infinispan.interceptors.InvocationContextInterceptor - ISPN000136: Error executing command PutKeyValueCommand, writing keys [128738+++++++++++++++++++++++++++++++++]
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-03-09 07:29:02,111 [remote-thread--p2-t4] WARN org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler - ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='EdiCache', command=PutKeyValueCommand{key=128738+++++++++++++++++++++++++++++++++, value=ImBusinessUnitRootEdi [address=128738, qualifier=null, buId=GC00029665M1], flags=[IGNORE_RETURN_VALUES], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true}}
> java.lang.NullPointerException
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8$LIRSEvictionPolicy.findIfEntriesNeedEvicting(BoundedEquivalentConcurrentHashMapV8.java:1532)
> at org.infinispan.commons.util.concurrent.jdk8backported.BoundedEquivalentConcurrentHashMapV8.compute(BoundedEquivalentConcurrentHashMapV8.java:3658)
> at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:226)
> at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:168)
> at org.infinispan.statetransfer.CommitManager.commit(CommitManager.java:98)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$InvalidationLogic.commitSingleEntry(ClusteringDependentLogic.java:377)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$ReplicationLogic.commitSingleEntry(ClusteringDependentLogic.java:421)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:114)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:478)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:654)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:455)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:529)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:560)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:198)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:93)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:62)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:352)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:290)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:107)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:107)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:43)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:43)
> at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years
[JBoss JIRA] (ISPN-6350) Data race in the ShardIndexManager under topology changes
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-6350?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes commented on ISPN-6350:
-----------------------------------------
One way to solve the issue would be to avoid closing the index immediately when notification of topology change arrives; instead, issue a delayed index close command to Hibernate Search backend, for example closeAfterCommit(), so that the lock can be released after the the current writing finishes.
[~sannegrinovero] proposed another approach:
When the listener arrives with topology changes, figure out if the index manager needs to close the indexwriter, by inspecting the new ownership of the segment.
* In case ownership does not change, keep the lock open
* If the segment ownership is moving to another node:
** Start forwarding the index works to the new owner, and issue a flushAndClose to the local index manager. A flushAndClose will have the same effect as the closeAfterCommit() described earlier, causing the index to be closed after the pending work finishes
*** On the node that is receiving the forwarded index works: wait for the lock to be available
**** Once available, applies all the index changes
**** If a timeout happens \[1\], forcefully acquires the lock \[2\]
Open questions:
\[1\] how long a timeout should be? It should be bigger than the time the original node takes to finish its pending work otherwise index corruption might happen.
\[2\] What if the node dies while still holding non-applied changes to the index?
> Data race in the ShardIndexManager under topology changes
> ---------------------------------------------------------
>
> Key: ISPN-6350
> URL: https://issues.jboss.org/browse/ISPN-6350
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 8.2.0.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Labels: affinity
>
> The following example data race can cause unrecoverable errors during indexing:
> \[node1\] cache.put(key) // key maps to segment 48, owned by node1
> \[node1\] starts shard 48
> \[node1\] acquires lock on shard 48
> \[node1\] starts writing to the index
> \[node1\] notification of topology changed, lock released on shard 48
> \[node1\] lock reacquired (still writing to the index)
> \[node1\] commit on shard 48
> \[node1\] shard still locked
> \[node2\] cache.put(key) // Node2 now owns segment 48
> \[node2\] starts shard 48
> \[node2\] tries to acquire the lock on shard 48
> \[node2\] fail (lock still owned by node1)
> The current mechanism employed by the {{ShardIndexManager}} during topology changes involves using a listener and closing the IndexWriter on all nodes upon ownership changes, so that the lock is released and can be reacquired by the new owner (1 segment maps to 1 shard).
> Since writing to a shard can take some time, the listener can be triggered in the middle of an index operation and the closing of the index writer will have a very short duration because it is sudden reacquired, and not released anymore.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years