Dan Berindei created ISPN-1839: ---------------------------------- Summary: L1 invalidation fails when some of the requestors have left the cluster Key: ISPN-1839 URL: https://issues.jboss.org/browse/ISPN-1839 Project: Infinispan Issue Type: Bug Components: Distributed Cache Affects Versions: 5.1.0.FINAL Reporter: Dan Berindei Assignee: Dan Berindei Fix For: 5.1.1.FINAL When a node leaves the cluster, it remains in the list of requestors for all the entries that it retrieved from other nodes. When those entries are updated, their owners try to invalidate them on the leaver. The invalidation command fails because the target has left, and this causes the entire put operation to fail. This can be seen in some of the HotRod server tests, e.g. HotRodDistributionTest: {noformat} 2012-02-02 10:56:06,186 TRACE (CacheViewTrigger,HotRodDistributionTest-NodeA-37471) [org.infinispan.cacheviews.PendingCacheViewChanges] Previous members are [HotRodDistributionTest-NodeA-37471, HotRodDistributionTest-NodeB-63614, HotRodDistributionTest-NodeC-41658], joiners are [], leavers are [HotRodDistributionTest-NodeC-41658], recovered after merge = false 2012-02-02 10:56:06,296 TRACE (OOB-2,ISPN,HotRodDistributionTest-NodeB-63614) [org.infinispan.distribution.L1ManagerImpl] Keys [ByteArrayKey{data=ByteArray{size=39, hashCode=7ec14a15, array=0x6b2d746573744469..}}] needs invalidation on [HotRodDistributionTest-NodeC-41658] 2012-02-02 10:56:06,296 TRACE (OOB-2,ISPN,HotRodDistributionTest-NodeB-63614) [org.infinispan.remoting.rpc.RpcManagerImpl] HotRodDistributionTest-NodeB-63614 invoking in future call InvalidateL1Command{num keys=1, forRehash=false, origin=HotRodDistributionTest-NodeA-37471} to recipient list [HotRodDistributionTest-NodeC-41658] 2012-02-02 10:56:06,297 TRACE (transport-thread-64) [org.infinispan.remoting.rpc.RpcManagerImpl] HotRodDistributionTest-NodeB-63614 broadcasting call InvalidateL1Command{num keys=1, forRehash=false, origin=HotRodDistributionTest-NodeA-37471} to recipient list [HotRodDistributionTest-NodeC-41658] 2012-02-02 10:56:06,298 ERROR (OOB-2,ISPN,HotRodDistributionTest-NodeB-63614) [org.infinispan.interceptors.InvocationContextInterceptor] ISPN000136: Execution error java.util.concurrent.ExecutionException: org.infinispan.remoting.transport.jgroups.SuspectException: One or more nodes have left the cluster while replicating command SingleRpcCommand{cacheName='hotRodDistSync', command=InvalidateL1Command{num keys=1, forRehash=false, origin=HotRodDistributionTest-NodeA-37471}} {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira