Write operations in invalidation mode can fail with OutdatedTopologyException ----------------------------------------------------------------------------- Key: ISPN-4016 URL: https://issues.jboss.org/browse/ISPN-4016 Project: Infinispan Issue Type: Bug Components: State Transfer Affects Versions: 6.0.1.Final, 7.0.0.Alpha1, 7.0.0.Alpha2, 7.0.0.Alpha3 Reporter: Dan Berindei Priority: Critical Labels: 630 Attachments: NonTxStateTransferInvalidationTest.log.zip I introduced this problem with the fix for ISPN-3873. Invalidation commands now have a topology id, so they can wait for the initial topology to be installed on a joiner. However, that means EntryWrappingInterceptor also checks the topology id, and if it has changed it will throw an OutdatedTopologyException. The exception is propagated all the way to the caller. OutdatedTopologyExceptions are not useful in invalidation mode, since the invalidation is always sent to the entire cluster. So EntryWrappingInterceptor should ignore the topology id in invalidation mode. {noformat} Tests run: 4052, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 313.505 sec <<< FAILURE!testInvalidationDuringStateTransfer(org.infinispan.statetransfer.NonTxStateTransferInvalidationTest) Time elapsed: 0.004 sec <<< FAILURE!java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NonTxStateTransferInvalidationTest-NodeB-5833, see cause for remote stack trace at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:202) at org.infinispan.commons.util.concurrent.NotifyingFutureImpl.get(NotifyingFutureImpl.java:84) at org.infinispan.statetransfer.NonTxStateTransferInvalidationTest.testInvalidationDuringStateTransfer(NonTxStateTransferInvalidationTest.java:115) Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NonTxStateTransferInvalidationTest-NodeB-5833, see cause for remote stack trace at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41) at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:66) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:547) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:280) at org.infinispan.interceptors.InvalidationInterceptor.invalidateAcrossCluster(InvalidationInterceptor.java:227) at org.infinispan.interceptors.InvalidationInterceptor.handleInvalidate(InvalidationInterceptor.java:143) at org.infinispan.interceptors.InvalidationInterceptor.visitPutKeyValueCommand(InvalidationInterceptor.java:80) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:326) at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:407) at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:164) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:68) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148) at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1403) at org.infinispan.CacheImpl.putInternal(CacheImpl.java:881) at org.infinispan.CacheImpl.access$100(CacheImpl.java:106) at org.infinispan.CacheImpl$2.call(CacheImpl.java:1015) ... 4 more Caused by: org.infinispan.statetransfer.OutdatedTopologyException: Cache topology changed while the command was executing: expected 2, got 3 at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:347) at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:407) at org.infinispan.interceptors.EntryWrappingInterceptor.visitInvalidateCommand(EntryWrappingInterceptor.java:139) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitInvalidateCommand(AbstractLockingInterceptor.java:87) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73) at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333) at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:39) at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48) at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95) at org.infinispan.remoting.InboundInvocationHandlerImpl.access$000(InboundInvocationHandlerImpl.java:50) at org.infinispan.remoting.InboundInvocationHandlerImpl$2.run(InboundInvocationHandlerImpl.java:178) ... 3 moreResults : Failed tests: NonTxStateTransferInvalidationTest.testInvalidationDuringStateTransfer:115 » Execution {noformat}