Dan Berindei created ISPN-5532: ---------------------------------- Summary: HotRod client retries on the same server after receiving an IllegalLifecycleStateException Key: ISPN-5532 URL: https://issues.jboss.org/browse/ISPN-5532 Project: Infinispan Issue Type: Bug Components: Remote Protocols, Test Suite - Server Affects Versions: 8.0.0.Alpha1, 7.2.2.Final Reporter: Dan Berindei Priority: Blocker Fix For: 8.0.0.Alpha2 When it receives an IllegalLifecycleStateException, the HotRod client retries the operation, but it may do so on the same server and it may have to retry again. Since the ISPN-5477 fix, {{DistTopologyChangeUnderLoadTest}} uses only one retry, and the test will fail if that retry happens to be on the same server. It would be better to exclude the server that returned the error when retrying. {noformat} java.util.concurrent.ExecutionException: org.infinispan.client.hotrod.exceptions.RemoteIllegalLifecycleStateException:Request for message id[9390] returned server error (status=0x88): org.infinispan.IllegalLifecycleStateException: ISPN000323: Default cache is in 'TERMINATED' state and so it does not accept new invocations. Either restart it or recreate the cache container. at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.infinispan.client.hotrod.DistTopologyChangeUnderLoadTest.testPutsSucceedWhileTopologyChanges(DistTopologyChangeUnderLoadTest.java:67) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:348) at org.testng.SuiteRunner.access$000(SuiteRunner.java:38) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382) at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.infinispan.client.hotrod.exceptions.RemoteIllegalLifecycleStateException:Request for message id[9390] returned server error (status=0x88): org.infinispan.IllegalLifecycleStateException: ISPN000323: Default cache is in 'TERMINATED' state and so it does not accept new invocations. Either restart it or recreate the cache container. at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:340) at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:126) at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:112) at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56) at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:57) at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:31) at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:20) at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:52) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:247) at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79) at org.infinispan.client.hotrod.DistTopologyChangeUnderLoadTest$PutHammer.call(DistTopologyChangeUnderLoadTest.java:80) at org.infinispan.client.hotrod.DistTopologyChangeUnderLoadTest$PutHammer.call(DistTopologyChangeUnderLoadTest.java:72) at org.infinispan.test.AbstractInfinispanTest$LoggingCallable.call(AbstractInfinispanTest.java:435) {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)