1:10 p.m.
add a retry to hotrod operations
---------------------------------
Key: ISPN-488
URL: https://jira.jboss.org/browse/ISPN-488
Project: Infinispan
Issue Type: Feature Request
Components: Cache Server
Affects Versions: 4.1.0.BETA2, 4.1.0.BETA1
Reporter: Mircea Markus
Assignee: Manik Surtani
Fix For: 4.1.0.CR1
From an email from Galder:
Hi Mircea,
I'm playing around with Hot Rod client/server for JUDCon demo and I've
found this. The demo consists of:
1. Start a server
2. Connect with client and do a put
3. Make several get calls on that server
4. Start a new node, clustered with replication and state transfer
on.
5. Make several calls, the calls are being load balanced in round
robin fashion.
6. Shutdown the first node and make a get call. This throws:
Exception in thread "main" HotRodServerException{messageId=0,
errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException:
java.io.IOException: Broken pipe
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:57)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsHelper.writeHeader(HotRodOperationsHelper.java:30)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.sendKeyOperation(HotRodOperationsImpl.java:256)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:39)
at
org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:19)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at java.nio.channels.Channels.write(Channels.java:60)
at java.nio.channels.Channels.access$000(Channels.java:47)
at java.nio.channels.Channels$1.write(Channels.java:134)
at java.io.OutputStream.write(OutputStream.java:58)
at java.nio.channels.Channels$1.write(Channels.java:115)
at
org.infinispan.io.UnsignedNumeric.writeUnsignedLong(UnsignedNumeric.java:134)
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:46)
... 7 more
Shouldn't have this been swallowed by the client and shouldn't have it
tried on the other node?
This happens if a server shuts down. After a failure the connection is cleaned.
A workaround for this should be "testOnBorrow" which would first test the
connection when fetching it from the server.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:33 p.m.
New subject: [JBoss JIRA] Commented: (ISPN-488) add a retry to hotrod operations
[
https://jira.jboss.org/browse/ISPN-488?page=com.atlassian.jira.plugin.sys...
]
Galder Zamarreno commented on ISPN-488:
---------------------------------------
Hmmm, not sure testOnBorrow is working. I'm getting this instead now:
Jun 9, 2010 1:32:29 PM org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport
<init>
SEVERE: Could not connect to server: /127.0.0.1:11311
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:34)
at
org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:31)
at
org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1179)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:164)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:92)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:37)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:24)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Jun 9, 2010 1:32:29 PM org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory
borrowTransportFromPool
SEVERE: Could not fetch transport
HotRodServerException{messageId=0, errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException: Could not connect to server:
/127.0.0.1:11311
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:40)
at
org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:31)
at
org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1179)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:164)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:92)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:37)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:24)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:34)
... 8 more
Exception in thread "main" HotRodServerException{messageId=0, errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException: Could not fetch transport
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:168)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:92)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:37)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:24)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: HotRodServerException{messageId=0, errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException: Could not connect to server:
/127.0.0.1:11311
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:40)
at
org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:31)
at
org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1179)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:164)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:34)
... 8 more
add a retry to hotrod operations
---------------------------------
Key: ISPN-488
URL: https://jira.jboss.org/browse/ISPN-488
Project: Infinispan
Issue Type: Feature Request
Components: Cache Server
Affects Versions: 4.1.0.BETA1, 4.1.0.BETA2
Reporter: Mircea Markus
Assignee: Manik Surtani
Fix For: 4.1.0.CR1
From an email from Galder:
Hi Mircea,
I'm playing around with Hot Rod client/server for JUDCon demo and I've
found this. The demo consists of:
1. Start a server
2. Connect with client and do a put
3. Make several get calls on that server
4. Start a new node, clustered with replication and state transfer
on.
5. Make several calls, the calls are being load balanced in round
robin fashion.
6. Shutdown the first node and make a get call. This throws:
Exception in thread "main" HotRodServerException{messageId=0,
errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException:
java.io.IOException: Broken pipe
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:57)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsHelper.writeHeader(HotRodOperationsHelper.java:30)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.sendKeyOperation(HotRodOperationsImpl.java:256)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:39)
at
org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:19)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at java.nio.channels.Channels.write(Channels.java:60)
at java.nio.channels.Channels.access$000(Channels.java:47)
at java.nio.channels.Channels$1.write(Channels.java:134)
at java.io.OutputStream.write(OutputStream.java:58)
at java.nio.channels.Channels$1.write(Channels.java:115)
at
org.infinispan.io.UnsignedNumeric.writeUnsignedLong(UnsignedNumeric.java:134)
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:46)
... 7 more
Shouldn't have this been swallowed by the client and shouldn't have it
tried on the other node?
This happens if a server shuts down. After a failure the connection is cleaned.
A workaround for this should be "testOnBorrow" which would first test the
connection when fetching it from the server.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:21 p.m.
New subject: [JBoss JIRA] Assigned: (ISPN-488) add a retry to hotrod operations
[
https://jira.jboss.org/browse/ISPN-488?page=com.atlassian.jira.plugin.sys...
]
Manik Surtani reassigned ISPN-488:
----------------------------------
Assignee: Mircea Markus (was: Manik Surtani)
add a retry to hotrod operations
---------------------------------
Key: ISPN-488
URL: https://jira.jboss.org/browse/ISPN-488
Project: Infinispan
Issue Type: Feature Request
Components: Cache Server
Affects Versions: 4.1.0.BETA1, 4.1.0.BETA2
Reporter: Mircea Markus
Assignee: Mircea Markus
Fix For: 4.1.0.CR1
From an email from Galder:
Hi Mircea,
I'm playing around with Hot Rod client/server for JUDCon demo and I've
found this. The demo consists of:
1. Start a server
2. Connect with client and do a put
3. Make several get calls on that server
4. Start a new node, clustered with replication and state transfer
on.
5. Make several calls, the calls are being load balanced in round
robin fashion.
6. Shutdown the first node and make a get call. This throws:
Exception in thread "main" HotRodServerException{messageId=0,
errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException:
java.io.IOException: Broken pipe
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:57)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsHelper.writeHeader(HotRodOperationsHelper.java:30)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.sendKeyOperation(HotRodOperationsImpl.java:256)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:39)
at
org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:19)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at java.nio.channels.Channels.write(Channels.java:60)
at java.nio.channels.Channels.access$000(Channels.java:47)
at java.nio.channels.Channels$1.write(Channels.java:134)
at java.io.OutputStream.write(OutputStream.java:58)
at java.nio.channels.Channels$1.write(Channels.java:115)
at
org.infinispan.io.UnsignedNumeric.writeUnsignedLong(UnsignedNumeric.java:134)
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:46)
... 7 more
Shouldn't have this been swallowed by the client and shouldn't have it
tried on the other node?
This happens if a server shuts down. After a failure the connection is cleaned.
A workaround for this should be "testOnBorrow" which would first test the
connection when fetching it from the server.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:46 a.m.
New subject: [JBoss JIRA] Work started: (ISPN-488) add a retry to hotrod operations
[
https://jira.jboss.org/browse/ISPN-488?page=com.atlassian.jira.plugin.sys...
]
Work on ISPN-488 started by Mircea Markus.
add a retry to hotrod operations
---------------------------------
Key: ISPN-488
URL: https://jira.jboss.org/browse/ISPN-488
Project: Infinispan
Issue Type: Feature Request
Components: Cache Server
Affects Versions: 4.1.0.BETA1, 4.1.0.BETA2
Reporter: Mircea Markus
Assignee: Mircea Markus
Fix For: 4.1.0.CR1
From an email from Galder:
Hi Mircea,
I'm playing around with Hot Rod client/server for JUDCon demo and I've
found this. The demo consists of:
1. Start a server
2. Connect with client and do a put
3. Make several get calls on that server
4. Start a new node, clustered with replication and state transfer
on.
5. Make several calls, the calls are being load balanced in round
robin fashion.
6. Shutdown the first node and make a get call. This throws:
Exception in thread "main" HotRodServerException{messageId=0,
errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException:
java.io.IOException: Broken pipe
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:57)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsHelper.writeHeader(HotRodOperationsHelper.java:30)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.sendKeyOperation(HotRodOperationsImpl.java:256)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:39)
at
org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:19)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at java.nio.channels.Channels.write(Channels.java:60)
at java.nio.channels.Channels.access$000(Channels.java:47)
at java.nio.channels.Channels$1.write(Channels.java:134)
at java.io.OutputStream.write(OutputStream.java:58)
at java.nio.channels.Channels$1.write(Channels.java:115)
at
org.infinispan.io.UnsignedNumeric.writeUnsignedLong(UnsignedNumeric.java:134)
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:46)
... 7 more
Shouldn't have this been swallowed by the client and shouldn't have it
tried on the other node?
This happens if a server shuts down. After a failure the connection is cleaned.
A workaround for this should be "testOnBorrow" which would first test the
connection when fetching it from the server.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:40 a.m.
New subject: [JBoss JIRA] Resolved: (ISPN-488) add a retry to hotrod operations
[
https://jira.jboss.org/browse/ISPN-488?page=com.atlassian.jira.plugin.sys...
]
Mircea Markus resolved ISPN-488.
--------------------------------
Resolution: Done
solved, unit test added as well (see retry package).
add a retry to hotrod operations
---------------------------------
Key: ISPN-488
URL: https://jira.jboss.org/browse/ISPN-488
Project: Infinispan
Issue Type: Feature Request
Components: Cache Server
Affects Versions: 4.1.0.BETA1, 4.1.0.BETA2
Reporter: Mircea Markus
Assignee: Mircea Markus
Fix For: 4.1.0.CR1
From an email from Galder:
Hi Mircea,
I'm playing around with Hot Rod client/server for JUDCon demo and I've
found this. The demo consists of:
1. Start a server
2. Connect with client and do a put
3. Make several get calls on that server
4. Start a new node, clustered with replication and state transfer
on.
5. Make several calls, the calls are being load balanced in round
robin fashion.
6. Shutdown the first node and make a get call. This throws:
Exception in thread "main" HotRodServerException{messageId=0,
errorStatusCode=0}
org.infinispan.client.hotrod.exceptions.TransportException:
java.io.IOException: Broken pipe
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48)
at
org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:57)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsHelper.writeHeader(HotRodOperationsHelper.java:30)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.sendKeyOperation(HotRodOperationsImpl.java:256)
at
org.infinispan.client.hotrod.impl.protocol.HotRodOperationsImpl.get(HotRodOperationsImpl.java:39)
at
org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:276)
at replication.GetOnReadLoop$.main(GetOnReadLoop.scala:19)
at replication.GetOnReadLoop.main(GetOnReadLoop.scala)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at java.nio.channels.Channels.write(Channels.java:60)
at java.nio.channels.Channels.access$000(Channels.java:47)
at java.nio.channels.Channels$1.write(Channels.java:134)
at java.io.OutputStream.write(OutputStream.java:58)
at java.nio.channels.Channels$1.write(Channels.java:115)
at
org.infinispan.io.UnsignedNumeric.writeUnsignedLong(UnsignedNumeric.java:134)
at
org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:46)
... 7 more
Shouldn't have this been swallowed by the client and shouldn't have it
tried on the other node?
This happens if a server shuts down. After a failure the connection is cleaned.
A workaround for this should be "testOnBorrow" which would first test the
connection when fetching it from the server.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5056
days inactive
5077
days old
infinispan-issues@lists.jboss.org
4 comments
3 participants
participants (3)
-
Galder Zamarreno (JIRA) -
Manik Surtani (JIRA)
-
Mircea Markus (JIRA)