[infinispan-issues] [JBoss JIRA] (ISPN-1630) KeyAffinityService's topology change listener can hang state transfer/cache view installation

[Galder Zamarreño (Updated) (JIRA)]

     [ https://issues.jboss.org/browse/ISPN-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Galder Zamarreño updated ISPN-1630:
-----------------------------------

               Description: 
When a node leaves, {{KeyGeneratorWorker}} can get in an infinite loop because the consistent hash doesn't return the leaver any more and {{KeyGeneratorWorker}} will never be able to generate the required number of keys.

{{KeyAffinityServiceImpl}} installs a listener for {{TopologyChangeEvents}} that is supposed to adjust the queues and prevent this problem. However, it doesn't work if {{KeyGeneratorWorker}} is already generating keys, because it needs to acquire the {{maxNumberInvariant}} write lock and KeyGeneratorWorker is never going to release the {{maxNumberInvariant}} read lock.

What's worse, because the listener is synchronous, it also blocks the cache view installation and other cluster changes will be ignored.
    Workaround Description:   (was: When a node leaves, {{KeyGeneratorWorker}} can get in an infinite loop because the consistent hash doesn't return the leaver any more and {{KeyGeneratorWorker}} will never be able to generate the required number of keys.

{{KeyAffinityServiceImpl}} installs a listener for {{TopologyChangeEvents}} that is supposed to adjust the queues and prevent this problem. However, it doesn't work if {{KeyGeneratorWorker}} is already generating keys, because it needs to acquire the {{maxNumberInvariant}} write lock and KeyGeneratorWorker is never going to release the {{maxNumberInvariant}} read lock.

What's worse, because the listener is synchronous, it also blocks the cache view installation and other cluster changes will be ignored.)

    
> KeyAffinityService's topology change listener can hang state transfer/cache view installation
> ---------------------------------------------------------------------------------------------
>
>                 Key: ISPN-1630
>                 URL: https://issues.jboss.org/browse/ISPN-1630
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Distributed Cache
>    Affects Versions: 5.1.0.CR1
>            Reporter: Dan Berindei
>            Assignee: Dan Berindei
>            Priority: Critical
>             Fix For: 5.1.0.CR2, 5.1.0.FINAL
>
>
> When a node leaves, {{KeyGeneratorWorker}} can get in an infinite loop because the consistent hash doesn't return the leaver any more and {{KeyGeneratorWorker}} will never be able to generate the required number of keys.
> {{KeyAffinityServiceImpl}} installs a listener for {{TopologyChangeEvents}} that is supposed to adjust the queues and prevent this problem. However, it doesn't work if {{KeyGeneratorWorker}} is already generating keys, because it needs to acquire the {{maxNumberInvariant}} write lock and KeyGeneratorWorker is never going to release the {{maxNumberInvariant}} read lock.
> What's worse, because the listener is synchronous, it also blocks the cache view installation and other cluster changes will be ignored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira