[infinispan-issues] [JBoss JIRA] Commented: (ISPN-1085) TopologyAwareConsistentHash can run in an endless loop when rehashing

[Jürgen Kellerer (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12599900#comment-12599900 ] 

Jürgen Kellerer commented on ISPN-1085:
---------------------------------------

I need to add the _getOwners(...)_ was called through "getStateProvidersOnLeave()" which does not protect _replCount_ from being greater than _caches.size();_

Also I discovered that either my setup is somewhat broken or another timing issue occurs as I have exceptions like "NoSuchElementException" on the attempt to access data immediately after the node came up caused by the fact that _caches_ and _positions_ is empty. The exception I got was:
{code} 
Caused by: java.util.NoSuchElementException
	at java.util.TreeMap.key(TreeMap.java:1223)
	at java.util.TreeMap.firstKey(TreeMap.java:284)
	at org.infinispan.distribution.ch.TopologyAwareConsistentHash.getOwner(TopologyAwareConsistentHash.java:141)
	at org.infinispan.distribution.ch.TopologyAwareConsistentHash.locate(TopologyAwareConsistentHash.java:53)
	at org.infinispan.distribution.ch.AbstractConsistentHash.isKeyLocalToAddress(AbstractConsistentHash.java:56)
	at org.infinispan.distribution.DistributionManagerImpl.getLocality(DistributionManagerImpl.java:360)
	at org.infinispan.interceptors.IsMarshallableInterceptor.getMightGoRemote(IsMarshallableInterceptor.java:133)
	at org.infinispan.interceptors.IsMarshallableInterceptor.visitGetKeyValueCommand(IsMarshallableInterceptor.java:70)
	at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:64)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:76)
	at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:88)
	at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:64)
	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:273)
	at org.infinispan.CacheDelegate.get(CacheDelegate.java:209)
{code}

Note: Using "DefaultConsistentHash" this does not happen as the method _isKeyLocalToAddress_ returns 'false' on empty _positions_ instead of failing.

> TopologyAwareConsistentHash can run in an endless loop when rehashing
> ---------------------------------------------------------------------
>
>                 Key: ISPN-1085
>                 URL: https://issues.jboss.org/browse/ISPN-1085
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Distributed Cache
>    Affects Versions: 4.2.1.FINAL
>         Environment: Infinispan 4.2.1 using distributed caches and 3 cluster nodes with siteId and machineId (siteId is the same on all nodes, machineId is different)
>            Reporter: Jürgen Kellerer
>            Assignee: Mircea Markus
>
> I'm not exactly sure how to reproduce it and why it happened but at least the problem is a quality / error handling issue. 
> TopologyAwareConsistentHash can run in an endless loop when rehashing runs and 'numOwners' is greater than the available addresses in 'processSequence'.
> The endless loop is in the method _getOwners()_ and the reason for it is shown in the simplified snippet below. It would really be helpful if the method either throws an exception with an exact definition what went wrong (e.g. including the address list and numOwners count) or handle the case if it's normal that this can happen when the cluster is just about to be built up.
> {code:java}
> private List<Address> getOwners(Address address, int numOwners) {
>    ...
>    ArrayList<Address> processSequence = ...
>    ...
>    List<Address> result = new ArrayList<Address>();
>    while (result.size() < numOwners) {
>        Iterator<Address> addrIt = processSequence.iterator();
>        while (addrIt.hasNext()) {
>           result.add(addrIt.next());
>           addrIt.remove();
>        }
>    }
>    return result;
> }
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira