<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 7 Jun 2011, at 13:13, Sanne Grinovero wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hello all,<br>in this scenario we have the Infinispan Lucene Directory using<br>batching (DummyTransaction), eviction and passivation to keep the<br>amount of memory being used for the index under control; I'm using<br>LIRS but experienced the same issue with all other strategies.<br><br>As you can see from the following stacktrace, the batching ends by<br>sending a commit request, so the status of the transaction is 8<br>(STATUS_COMMITTING) in this context.<br>The new data is stored in the DataContainer, then the<br>BoundedConcurrentHashMap notifies the EvictionManagerImpl as it has to<br>evict some values, and this one attempts to acquire a lock on the<br>to-be-evicted keys (which are obviously not the same I'm trying to<br>store).<br>Acquiring this lock is an invalid operation as the transaction is in<br>commit state, and so this operation fails with an exception.<br><br>Thread [Hibernate Search: Directory writer-1] (Suspended (breakpoint<br>at line 92 in LockManagerImpl))<br><span class="Apple-tab-span" style="white-space:pre"> </span>LockManagerImpl.lockAndRecord(Object, InvocationContext) line: 92<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>EvictionManagerImpl.acquireLock(InvocationContext, Object) line: 210<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>EvictionManagerImpl.onEntryEviction(Object, InternalCacheEntry) line: 170<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>EvictionManagerImpl.onEntryEviction(Map<Object,InternalCacheEntry>) line: 162<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>DefaultDataContainer$DefaultEvictionListener.onEntryEviction(Map<Object,InternalCacheEntry>)<br>line: 201<br><span class="Apple-tab-span" style="white-space:pre"> </span>BoundedConcurrentHashMap$Segment<K,V>.notifyEvictionListener(Set<HashEntry<K,V>>)<br>line: 1176<br><span class="Apple-tab-span" style="white-space:pre"> </span>BoundedConcurrentHashMap$Segment<K,V>.put(K, int, V, boolean) line: 1011<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>BoundedConcurrentHashMap<K,V>.put(K, V) line: 1556<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>DefaultDataContainer.put(Object, Object, long, long) line: 148<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>ReadCommittedEntry.commit(DataContainer) line: 177<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>LockingInterceptor.commitEntry(CacheEntry, boolean) line: 389<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>LockingInterceptor.cleanupLocks(InvocationContext, boolean) line: 367<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>LockingInterceptor.visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 98<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheStoreInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheStoreInterceptor.visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 148<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheLoaderInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheLoaderInterceptor(CommandInterceptor).handleDefault(InvocationContext,<br>VisitableCommand) line: 133<br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheLoaderInterceptor(AbstractVisitor).visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 116<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>NotificationInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>NotificationInterceptor.visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 56<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>TxInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>TxInterceptor.visitCommitCommand(TxInvocationContext, CommitCommand) line: 142<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheMgmtInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheMgmtInterceptor(CommandInterceptor).handleDefault(InvocationContext,<br>VisitableCommand) line: 133<br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheMgmtInterceptor(AbstractVisitor).visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 116<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>InvocationContextInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>InvocationContextInterceptor.handleAll(InvocationContext,<br>VisitableCommand) line: 96<br><span class="Apple-tab-span" style="white-space:pre"> </span>InvocationContextInterceptor.handleDefault(InvocationContext,<br>VisitableCommand) line: 63<br><span class="Apple-tab-span" style="white-space:pre"> </span>InvocationContextInterceptor(AbstractVisitor).visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 116<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchingInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext,<br>VisitableCommand) line: 119<br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchingInterceptor.handleDefault(InvocationContext,<br>VisitableCommand) line: 77<br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchingInterceptor(AbstractVisitor).visitCommitCommand(TxInvocationContext,<br>CommitCommand) line: 116<br><span class="Apple-tab-span" style="white-space:pre"> </span>CommitCommand.acceptVisitor(InvocationContext, Visitor) line: 60<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>InterceptorChain.invoke(InvocationContext, VisitableCommand) line: 274<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>TransactionCoordinator.commit(LocalTransaction, boolean) line: 136<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>TransactionXaAdapter.commit(Xid, boolean) line: 124<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>DummyTransaction.runCommitTx() line: 312<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>DummyTransaction.commit() line: 99<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchModeTransactionManager(DummyBaseTransactionManager).commit() line: 97<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchContainer.resolveTransaction(BatchContainer$BatchDetails,<br>boolean) line: 131<br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchContainer.endBatch(boolean, boolean) line: 108<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>BatchContainer.endBatch(boolean) line: 93<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>CacheDelegate<K,V>.endBatch(boolean) line: 436<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>InfinispanIndexOutput.close() line: 208<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>IOUtils.closeSafely(Closeable...) line: 80<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>FieldsWriter.close() line: 111<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>StoredFieldsWriter.flush(SegmentWriteState) line: 52<span class="Apple-tab-span" style="white-space:pre"> </span><br><span class="Apple-tab-span" style="white-space:pre"> </span>DocFieldProcessor.flush(Collection<DocConsumerPerThread>,<br>SegmentWriteState) line: 58<br><br>I would like to remove the lock operation from the eviction listener,<br>but I'm not understanding why this locking is needed there and would<br>appreciate some explanations or help with this.<br></div></blockquote>Looking at the code, my first thought was that it is needed for sync-ing cache store /passivator access on that key.</div><div>But afaik the cache store takes care of key locking at its own level[1], so I think that should be remove entirely.</div><div><br></div><div>[1] <a href="https://github.com/mmarkus/infinispan/blob/master/core/src/main/java/org/infinispan/loaders/LockSupportCacheStore.java">https://github.com/mmarkus/infinispan/blob/master/core/src/main/java/org/infinispan/loaders/LockSupportCacheStore.java</a></div><div><blockquote type="cite"><div>Shouldn't an evict operation be a "best effort" operation in all<br>cases,</div></blockquote>+1. Even if my statement above doesn't stand and there's a reason to keep the lock for consistency, it should be a best effort: try to lock with 0 timeout, if fails just ignore it and move on. <br><blockquote type="cite"><div> or is the idea here that we want the evictable data to be<br>consistently evicted on multiple nodes, or maybe even rollback an<br>evict operation?<br></div></blockquote>We don't advertise cluster-wise eviction consistency, so I doubt that is the case. <br><blockquote type="cite"><div><br>Cheers,<br>Sanne<br>_______________________________________________<br>infinispan-dev mailing list<br><a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>https://lists.jboss.org/mailman/listinfo/infinispan-dev<br></div></blockquote></div><br></body></html>