[infinispan-issues] [JBoss JIRA] Assigned: (ISPN-488) add a retry to hotrod operations

add a retry to hotrod operations --------------------------------- Key: ISPN-488 URL: https://jira.jboss.org/browse/ISPN-488 Project: Infinispan Issue Type: Feature Request Components: Cache Server Affects Versions: 4.1.0.BETA1, 4.1.0.BETA2 Reporter: Mircea Markus Assignee: Mircea Markus Fix For: 4.1.0.CR1 From an email from Galder: Hi Mircea, I'm playing around with Hot Rod client/server for JUDCon demo and I've found this. The demo consists of: 1. Start a server 2. Connect with client and do a put 3. Make several get calls on that server 4. Start a new node, clustered with replication and state transfer on. 5. Make several calls, the calls are being load balanced in round robin fashion. 6. Shutdown the first node and make a get call. This throws: Exception in thread "main" HotRodServerException{messageId=0, errorStatusCode=0} org.infinispan.client.hotrod.exceptions.TransportException: java.io.IOException: Broken pipe at org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48) at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:57) at org.infinispan.client.hotrod.impl.protocol.HotRodOperationsHelper.writeHeader(HotRodOperationsHelper.java:30) at org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.sendKeyOperation(HotRodOperationsImpl.java:256) at org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:39) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276) at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:19) at replication.GetOnReadLoop.main(GetOnReadLoop.scala) Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcher.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104) at sun.nio.ch.IOUtil.write(IOUtil.java:75) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334) at java.nio.channels.Channels.write(Channels.java:60) at java.nio.channels.Channels.access$000(Channels.java:47) at java.nio.channels.Channels$1.write(Channels.java:134) at java.io.OutputStream.write(OutputStream.java:58) at java.nio.channels.Channels$1.write(Channels.java:115) at org.infinispan.io.UnsignedNumeric.writeUnsignedLong(UnsignedNumeric.java:134) at org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:46) ... 7 more Shouldn't have this been swallowed by the client and shouldn't have it tried on the other node? This happens if a server shuts down. After a failure the connection is cleaned. A workaround for this should be "testOnBorrow" which would first test the connection when fetching it from the server.