Race condition in AbstractInvocationContextFactory (results in NullPointerException in EquivalentHashMap when receiving an invalidate command during startup) ------------------------------------------------------------------------------------------------------------------------------------------------------------- Key: ISPN-3873 URL: https://issues.jboss.org/browse/ISPN-3873 Project: Infinispan Issue Type: Bug Components: Core Affects Versions: 6.0.1.Final Environment: Ubuntu Linux 12.04_LTS, Oracle Java SE 1.7.0_45 Reporter: Karl von Randow Assignee: Dan Berindei Labels: 630 Fix For: 7.0.0.Alpha1, 7.0.0.Final Attachments: infinispan.trace.gz Sometimes when I start one server in a cluster I see the NullPointerException reproduced below. This only occurs if there is another server running. I run multiple caches with clustering using invalidation. JGroups using JDBC_PING and TCP. The caches are non-transactional. The NPE is because the keyEq (Equivalence instance for keys) is null. I believe this is because the command is processed before the start() method has been called on the AbstractInvocationContextFactory, as the start() method sets the keyEq ivar. At this point I'm not sure how to fix it! java.lang.NullPointerException at org.infinispan.commons.equivalence.EquivalentHashMap.put(EquivalentHashMap.java:176) at org.infinispan.commons.equivalence.EquivalentHashSet.add(EquivalentHashSet.java:109) at org.infinispan.context.impl.NonTxInvocationContext.addLockedKey(NonTxInvocationContext.java:84) at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:190) at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:181) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:149) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitInvalidateCommand(AbstractLockingInterceptor.java:85) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73) at org.infinispan.commands.AbstractVisitor.visitInvalidateCommand(AbstractVisitor.java:111) at org.infinispan.commands.write.InvalidateCommand.acceptVisitor(InvalidateCommand.java:118) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333) at org.infinispan.commands.remote.BaseRpcInvokingCommand.processVisitableCommand(BaseRpcInvokingCommand.java:39) at org.infinispan.commands.remote.SingleRpcCommand.perform(SingleRpcCommand.java:48) at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95) at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:186) at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:84) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:259) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:211) at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460) at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377) at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247) at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:665) at org.jgroups.JChannel.up(JChannel.java:730) at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1019) at org.jgroups.protocols.FRAG2.up(FRAG2.java:182) at org.jgroups.protocols.FlowControl.up(FlowControl.java:434) at org.jgroups.protocols.FlowControl.up(FlowControl.java:434) at org.jgroups.stack.Protocol.up(Protocol.java:409) at org.jgroups.stack.Protocol.up(Protocol.java:409) at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294) at org.jgroups.protocols.RSVP.up(RSVP.java:221) at org.jgroups.protocols.UNICAST2.removeAndPassUp(UNICAST2.java:919) at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:800) at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:415) at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600) at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147) at org.jgroups.protocols.FD.up(FD.java:255) at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:301) at org.jgroups.protocols.MERGE2.up(MERGE2.java:209) at org.jgroups.protocols.Discovery.up(Discovery.java:379) at org.jgroups.protocols.TP.passMessageUp(TP.java:1399) at org.jgroups.protocols.TP$MyHandler.run(TP.java:1585) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)