Let me further elaborate this issue:
In our application, there are some Message listeners running on the client side, these
message listeners will receive messages from queue/topic deployed in JBoss Messaging
Configuration:
We created our own JMS Connection factory which uses the default remoting connector. As
you know, the default remoting connector is configured to use the bisocket transport
During we run our application, we open the JBoss web console to monitor the value of
currentClientPoolSize under Jboss.remoting JMX MBean
How to reproduce this issue:
1. Run 5 message listeners in the client side to receive messages from JBoss Messaging,
then we monitor the value of currentClientPoolSize, we observe the value is 10
2. After processing several messages, we manually pull out the ethernet cable. After some
time, we found the value of currentClientPoolSize is still 10, no any change. (If we
killed the process of message listener in client side, then the value of
currentClientPoolSize will decrease to 0 immediately)
3. We run another 5 message listeners in client side, then the value of
currentClientPoolSize will become 20
4. After we do the same operations above several times, the value of currentClientPoolSize
will increase continuously. Once the value of currentClientPoolSize is equal to the
MaxPoolSize, then the subsequent incoming client requests will hang, and we will encounter
the following exception in server side
| 2009-10-20 18:08:09,655 ERROR [org.jboss.remoting.transport.socket.ServerThread]
Worker thread initi
| alization failure
| java.net.SocketException: Connection reset
| at java.net.SocketInputStream.read(SocketInputStream.java:168)
| at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
| at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
| at java.io.FilterInputStream.read(FilterInputStream.java:66)
| at
org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:859)
| at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:545)
| at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:406)
| at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
|
Conclusion: JBoss Messaging will not close the failure connections if they are caused by
manually pulling out ethernet cable. As a result, the value of currentClientPoolSize will
increase continuously and finally the new client requests will hang
We found several similar problems in JBoss JIRA as below:
https://jira.jboss.org/jira/browse/JBMESSAGING-1268
https://jira.jboss.org/jira/browse/JBREM-947
https://jira.jboss.org/jira/browse/JBREM-949
In the JBREM-947 and JBREM-949, the remoting wrongly depend on the socket timeout for
failure detection, the bug reporter said 'Remoting should not be dependent on the
socket timeout for failure detection, the connetion validation and socket timeout should
be possible to be configured separately'. According to the JIRA, this issue had been
fixed in Remoting 2.2.2. SP8 and 2.4.0.CR2 (Pinto), since it is said 'JBREM-947 - is
the proper fix in remoting which will allow us to configure timeouts and validation
interval separately'. (stated in JBMESSAGING-1268)
However, according to our test, this issue was not fixed properly. We suspect the Remoting
2.2.3 still wrongly depend on the socket timeout for failure connection detection. So, we
tried to change the timeout of bisocket transport from 0 (infinite) to a lower value e.g.
10 seconds, then we repeat the test steps again and observe the jboss web console,
fortunately, we found the failure connection will be closed after 10 seconds. In
JBREM-947, a daemon thread and validatorPingTimeout parameter are mentioned, we think
daemon thread should use parameter validatorPingPeriod and validatorPingTimeout to close
the failure connection instead of socket timeout. Thus, we post this to seek your
expertise. Please correct us if we are wrong.
View the original post :
http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4265001#...
Reply to the post :
http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&a...