[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño commented on ISPN-2172: ---------------------------------------- Example, k0: {code}[org.infinispan.interceptors.CallInterceptor] (main:___defaultcache) Executing command: RemoveCommand{key=k0, value=null, flags=null}. [org.infinispan.interceptors.DistributionInterceptor] (main:___defaultcache) Invoking command RemoveCommand{key=k0, value=null, flags=null} on hosts [NodeA-60583, NodeA-47444] [org.infinispan.interceptors.DistributionInterceptor] (main:___defaultcache) Put occuring on node, requesting L1 cache invalidation for keys [k0]. Other data owners are [NodeA-47444, NodeA-60583] [org.infinispan.distribution.L1ManagerImpl] (main:___defaultcache) Invalidating L1 caches for keys [k0] [org.infinispan.distribution.L1ManagerImpl] (main:___defaultcache) No L1 caches to invalidate{code} But it's not invalidating NodeB's cache: {code}java.lang.AssertionError: Key: k0 is present in cache at Cache '___defaultcache'@NodeB-58777{code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño commented on ISPN-2172: ---------------------------------------- The issue only happens when rehashing involved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-2172: ----------------------------------- Comment: was deleted (was: I think I've found the problem: L1ManagerImpl will only invalidate those keys that have been requested by others nodes (via a get()). However, if a node receives data as part of rehashing and it does not belong to it, it'd be added to L1 but it won't be registered as requestor, hence it won't be invalidated when the remove happens.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño edited comment on ISPN-2172 at 7/26/12 11:32 AM: ------------------------------------------------------------------ This is the problem: Node A pushes k1 to node B, and node A stops being an owner. In theory, node A should remove k1 from memory, but since L1 is enabled, it keeps it in memory, in the L1 (quite possibly for efficiency). However, node B has not been told that node A is gonna actually keep it (iow, node A has not been added as requestor for key k1 in node B's L1ManagerImpl), so when a remove comes to node B, it doesn't realise that it needs to remove from node A too. The reason node B does not add node A as requestor for k1 is cos the put call being passed up the stack when applying the state transfer is local, so the logic in DistributionInterceptor.visitPutKeyValueCommand() to add k1 as being 'requested' by node A does not kick in. was (Author: galder.zamarreno): This is the problem is this: node A pushes k1 to node B, and node A stops being an owner. In theory, node A should remove k1 from memory, but since L1 is enabled, it keeps it in memory, in the L1 (quite possibly for efficiency). However, node A has not been told that node B is gonna actually keep it (iow, k1 for node B has not been added to the L1ManagerImpl), so when a remove comes to node A, it doesn't realise that it needs to remove. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-2172: ----------------------------------- Labels: (was: jdg jdg6) Git Pull Request: https://github.com/infinispan/infinispan/pull/1230 (was: https://github.com/infinispan/infinispan/pull/1230) Workaround Description: Several workarounds: - Disabling L1 works. - Enabling L1, but disabling onRehash(). Finallt, data in L1 cache will eventually timeout, so not the end of the world if for a while the number of entries is non-zero. was: Disabling L1 works. Also, data in L1 cache will eventually timeout, so not the end of the world. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Manik Surtani commented on ISPN-2172: ------------------------------------- Does this take into account the new NBST that Dan's been working on? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño commented on ISPN-2172: ---------------------------------------- [~manik] He's on the loop, need to catch up with him on the pull req -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-2172: ----------------------------------- Assignee: Dan Berindei (was: Galder Zamarreño) Git Pull Request: https://github.com/infinispan/infinispan/pull/1230 (was: https://github.com/infinispan/infinispan/pull/1230) Assigning to Dan which will look into this in the light of NBST when he and Adrian re-implement the l1.onRehash accordingly. The pull request should still be left open in case there's further discussion and also provides with a unit test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-2172?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-2172: ------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Duplicate Issue Duplicate of ISPN-1830 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira