[infinispan-issues] [JBoss JIRA] (ISPN-9759) Hot Rod server non-hash aware topology updates can include non-members

[Dan Berindei (Jira)]

Dan Berindei created ISPN-9759:
----------------------------------

             Summary: Hot Rod server non-hash aware topology updates can include non-members
                 Key: ISPN-9759
                 URL: https://issues.jboss.org/browse/ISPN-9759
             Project: Infinispan
          Issue Type: Bug
          Components: Test Suite - Server
    Affects Versions: 9.4.2.Final
            Reporter: Dan Berindei
             Fix For: 10.0.0.Alpha2


When sending a non-hash aware topology update, the server includes all the servers that are present in the topology cache. This includes both servers that don't have the cache running yet (if the cache was started dynamically) and servers that are shutting down or already shutdown (because a node doesn't remove itself from the address cache before shutting down). 

When a node shut down, the remaining nodes eventually see the view change and remove the stopped server from the address cache, but likely after sending a topology update with the new topology id to the clients. In cases where a rebalance is not necessary (e.g. replicated caches, or a single node is alive), a corrected topology update is never sent to the client.

This is causing random failures in {{HotRod10ReplicationTest.testReplicatedPutWithTopologyChanges}},{{HotRodReplicationTest.testReplicatedPutWithTopologyChanges}} and {{HotRod12ReplicationTest.testReplicatedPutWithTopologyChanges}}

I saw it on a pull request build first, but I see it (and its subclasses) has been randomly failing in master as well. I am able to reliably reproduce the failure on my machine if I add a small delay in {{CrashedMemberDetectorListener.detectCrashedMember()}}.

{noformat}
16:24:18,288 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.server.hotrod.HotRod12ReplicationTest.testReplicatedPutWithTopologyChanges
java.lang.AssertionError: expected:<3> but was:<2>
	at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.9.9.jar:?]
	at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364) ~[testng-6.9.9.jar:?]
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80) ~[testng-6.9.9.jar:?]
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245) ~[testng-6.9.9.jar:?]
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252) ~[testng-6.9.9.jar:?]
	at org.infinispan.server.hotrod.HotRodReplicationTest.testReplicatedPutWithTopologyChanges(HotRodReplicationTest.java:145) ~[test-classes/:?]
{noformat}



--
This message was sent by Atlassian Jira
(v7.12.1#712002)