[ https://issues.jboss.org/browse/ISPN-1122?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-1122: ----------------------------------- Description: I've seen this in master, when running ServerErrorTest from testsuite: {code}[z@dnb-2:~/Go/code/infinispan.git]% (t_1119_m⚡) more client/hotrod-client/target/surefire-reports/TEST-org.infinispan.client.hotrod.ServerErrorTest-testErrorWhileDoingPut.xml <?xml version="1.0" encoding="UTF-8" ?> <testsuite time="0" name="org.infinispan.client.hotrod.ServerErrorTest-testErrorWhileDoingPut"> <testcase time="0.265" classname="org.infinispan.client.hotrod.ServerErrorTest" name="org.infinispan.client.hotrod.ServerErrorTest-testErrorWhileDoingPut"> <failure message="Invalid magic number. Expected 0xa1 and received 0x50" type="org.infinispan.client.hotrod.exceptions.InvalidResponseException"> org.infinispan.client.hotrod.exceptions.InvalidResponseException:: Invalid magic number. Expected 0xa1 and received 0x50 at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:107) at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:70) at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:49) at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:62) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:201) at org.infinispan.CacheSupport.put(CacheSupport.java:51) at org.infinispan.client.hotrod.ServerErrorTest.testErrorWhileDoingPut(ServerErrorTest.java:97) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:644) at org.testng.internal.Invoker.invokeMethod(Invoker.java:546) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:700) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1002) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:137) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:121) at org.testng.TestRunner.runWorkers(TestRunner.java:909) at org.testng.TestRunner.privateRun(TestRunner.java:618) at org.testng.TestRunner.run(TestRunner.java:499) at org.testng.SuiteRunner.runTest(SuiteRunner.java:332) at org.testng.SuiteRunner.access$000(SuiteRunner.java:33) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:358) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:142) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) </failure>{code} This needs investigate cos it could be related to other Invalid* issues QE have been encountering with EDG. was: I've seen this in master, when running ServerErrorTest from testsuite: {code}[z@dnb-2:~/Go/code/infinispan.git]% (t_1119_m⚡) more client/hotrod-client/target/surefire-reports/TEST-org.infinispan.client.hotrod.ServerErrorTest-testErrorWhileDoingPut.xml <?xml version="1.0" encoding="UTF-8" ?> <testsuite time="0" name="org.infinispan.client.hotrod.ServerErrorTest-testErrorWhileDoingPut"> <testcase time="0.265" classname="org.infinispan.client.hotrod.ServerErrorTest" name="org.infinispan.client.hotrod.ServerErrorTest-testErrorWhileDoingPut"> <failure message="Invalid magic number. Expected 0xa1 and received 0x50" type="org.infinispan.client.hotrod.exceptions.InvalidResponseException">org.infinispan.client.hotrod.exceptions.InvalidResponseException:: Invalid magic number. Expected 0xa1 and received 0x50 at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:107) at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:70) at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:49) at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:62) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:201) at org.infinispan.CacheSupport.put(CacheSupport.java:51) at org.infinispan.client.hotrod.ServerErrorTest.testErrorWhileDoingPut(ServerErrorTest.java:97) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:644) at org.testng.internal.Invoker.invokeMethod(Invoker.java:546) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:700) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1002) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:137) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:121) at org.testng.TestRunner.runWorkers(TestRunner.java:909) at org.testng.TestRunner.privateRun(TestRunner.java:618) at org.testng.TestRunner.run(TestRunner.java:499) at org.testng.SuiteRunner.runTest(SuiteRunner.java:332) at org.testng.SuiteRunner.access$000(SuiteRunner.java:33) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:358) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:142) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) </failure>{code} This needs investigate cos it could be related to other Invalid* issues QE have been encountering with EDG. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1122?page=com.atlassian.jira.plugin.... ] Galder Zamarreño commented on ISPN-1122: ---------------------------------------- Just been banging my head against it further via a similar issue, JBPAPP-6113, and the key might be the same. Sometimes, the response is dealt by a different thread to the one that sent the request, and when that happens, the next request by that tcp transport object goes bang! Btw, this cannot be replicated always... and this testsuite failure is a random one -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1122?page=com.atlassian.jira.plugin.... ] Galder Zamarreño commented on ISPN-1122: ---------------------------------------- Thanks to ISPN-1126 we can now see what happens in the previous request with this TCP tranport class. For example, sometimes a socket read timeout might be risen: {code}2011-05-20 12:48:39,602 212028 TRACE [org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation] (Runner - 0:) Exception encountered. Retry 1 out of 40 org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:156) at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:104) at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:70) at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:49) at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:62) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:200) at org.infinispan.CacheSupport.put(CacheSupport.java:50) at org.jboss.smartfrog.edg.adapter.HotRodAdapter$HotRodRemoteCacheAdapter.put(HotRodAdapter.java:321) at org.jboss.smartfrog.edg.loaddriver.CacheRequestProcessorFactory$InfinispanRequestProcessor.processRequest(CacheRequestProcessorFactory.java:169) at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:51) at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:87) at java.lang.Thread.run(Thread.java:662) Caused by: java.net.SocketTimeoutException at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read(BufferedInputStream.java:237) at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:154) {code} And in that case, the TCP transport class is sent back to the pool without marking it invalid. So, the next time this TCP transport instance is used, the data in socket might still belong to the previous request, and that's how you can end up with this type of Invalid* exceptions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1122?page=com.atlassian.jira.plugin.... ] Galder Zamarreño reassigned ISPN-1122: -------------------------------------- Assignee: Trustin Lee (was: Galder Zamarreño) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1122?page=com.atlassian.jira.plugin.... ] Trustin Lee commented on ISPN-1122: ----------------------------------- {{TcpTransport.isInvalid()}} only checks if the socket is closed or not. I added a boolean flag to mark it when an {{IOException}} is raised during I/O, hence properly handling non-fatal exceptions like {{SocketTimeoutException}} and {{InterruptedIOException}}. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1122?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-1122: ----------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira