3:32 a.m.
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public (Everyone can see)
Reporter: Tim Fox
Fix For: 2.2.2.SP7
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died -
i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire server
disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:32 a.m.
New subject: [JBoss JIRA] Updated: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=all ]
Tim Fox updated JBREM-947:
--------------------------
Fix Version/s: (was: 2.2.2.SP7)
Affects Version/s: 2.2.2.SP7
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:47 p.m.
New subject: [JBoss JIRA] Updated: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=all ]
Ron Sigal updated JBREM-947:
----------------------------
Fix Version/s: 2.4.0.GA
Assignee: Ron Sigal
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.GA
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:21 p.m.
New subject: [JBoss JIRA] Commented: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=comments#action_12409702 ]
Ron Sigal commented on JBREM-947:
---------------------------------
ConnectionValidator has been changed so that (1) the invocation whose result is expected
and (2) the test that the result has arrived are performed on separate threads. For
example, before ConnectionValidator.run() makes the invocation, it schedules a
WaitOnConnectionCheckTimerTask, which will wait for up to the configured timeout
milliseconds. One of two things will happen:
1. The invocation returns within the configured timeout window,
WaitOnConnectionCheckTimerTask is notified, and it quietly ends.
2. The invocation does not return within the configured timeout window (and possibly
does not return at all), in which case WaitOnConnectionCheckTimerTask informs all
listeners that a problem has been detected.
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.GA
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:21 p.m.
New subject: [JBoss JIRA] Commented: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=comments#action_12409703 ]
Ron Sigal commented on JBREM-947:
---------------------------------
There are no specific new unit tests. All existing unit tests should continue to pass.
Also, a manual test should be run that verifies that pulling the network cable results in
listeners being informed of a network problem before the invocation call times out.
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.GA
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:43 p.m.
New subject: [JBoss JIRA] Updated: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=all ]
Ron Sigal updated JBREM-947:
----------------------------
Fix Version/s: 2.4.0.CR2
(was: 2.4.0.GA)
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.CR2
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:18 a.m.
New subject: [JBoss JIRA] Closed: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=all ]
Ron Sigal closed JBREM-947.
---------------------------
Resolution: Done
With org.jboss.test.remoting.connection.ConnectionValidationServer running on one machine
and org.jboss.test.remoting.connection.ConnectionValidatorRemoteClient on the other,
pulling the plug on the network cable verifies that ConnectionValidator informs listeners
of a network problem. In particular, ConnectionValidatorRemoteClient sets
"timeout"=20000 and "validatorPingTimeout" to 1000, and
ConnectionValidator detects a network problem almost immediately after the cable is
unplugged.
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.CR2
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:56 a.m.
New subject: [JBoss JIRA] Commented: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=comments#action_12410538 ]
Tim Fox commented on JBREM-947:
-------------------------------
What are the performance implications of the way this is implemented.
Is a new timer task scheduled for *every* invocation? If so, this will be very slow
surely... ?
I couldn't see the actual fix on the svn commits tab.
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.CR2
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:21 p.m.
New subject: [JBoss JIRA] Commented: (JBREM-947) ConnectionValidator hangs when server dies
[ http://jira.jboss.com/jira/browse/JBREM-947?page=comments#action_12410631 ]
Ron Sigal commented on JBREM-947:
---------------------------------
Actually, it's a new daemon Thread for each ping. If this were on a critical path I
would have done something more clever, but we're talking about one new object every
few seconds, so I doubt that it will have much of an impact. Nonetheless, I'll do
some profiling before the final release to check for hot spots.
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: http://jira.jboss.com/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assigned To: Ron Sigal
Fix For: 2.4.0.CR2
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:09 a.m.
New subject: [JBoss JIRA] Commented: (JBREM-947) ConnectionValidator hangs when server dies
[
https://jira.jboss.org/jira/browse/JBREM-947?page=com.atlassian.jira.plug...
]
mingjun jiang commented on JBREM-947:
-------------------------------------
Accoring to this JIRA, It seems that this bug was fixed in 2.4.0 . However, in
https://jira.jboss.org/jira/browse/JBREM-949, it is said that this bug was fixed in
Remoting 2.2.2 SP8. I am quite confused. Now, we encountered the similar problem when we
are using 2.2.3 SP1.
The bisocket connection won't be closed if pulling out the ethernet cable between
client and server. It seems that the failure detection program won't close the failure
connection, as a result, the subsequent requests will hang after connection account
exceeds the threshold (MaxPoolSize).
I had opened a JIRA issue to report this problem:
https://jira.jboss.org/jira/browse/JBREM-1169, please refer to this JIRA link for more
details
I am adding such comments to seek your expertise, thanks
ConnectionValidator hangs when server dies
------------------------------------------
Key: JBREM-947
URL: https://jira.jboss.org/jira/browse/JBREM-947
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.2.SP7
Reporter: Tim Fox
Assignee: Ron Sigal (old account)
Fix For: 2.4.0.CR2 (Pinto)
If the connection between client and server is pulled (pull the real cable) or the entire
server suddenly dies, then the connection won't be closed from the server (unlike a
kill -9 of the server where the OS will terminate that processses connections), so the
client making the write() or read() on the socket won't receive an exception.
In the eyes of TCP the connection is still alive and the read/write will block until the
socket timeout is reached.
Typically the socket timeout will be much higher than the desired failure detection time
(the validation interval), but currently failure will never be detected in this situation
before the socket timeout time.
Remoting should not be dependent on the socket timeout for failure detection, the
connetion validation and socket timeout should be possible to be configured separately.
E.g. you might want to configure a socket timeout of 60 seconds, but a connection
validation frequency (ping) of 5 seconds. Currently this is not possible.
The current implementation gives inconsistent behaviour depending on how the server died
- i.e. whether the process died (e.g. kill -9) or the cable was pulled or the entire
server disappeared.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5295
days inactive
5896
days old
jboss-remoting-issues@lists.jboss.org
9 comments
3 participants
participants (3)
-
mingjun jiang (JIRA) -
Ron Sigal (JIRA)
-
Tim Fox (JIRA)