2:40 p.m.
Joseph S [http://community.jboss.org/people/jselvin] created the discussion
"Re: Sockets left in CLOSE_WAIT on server for JMS connections"
To view the discussion, visit: http://community.jboss.org/message/596026#596026
--------------------------------------------------------------
Hi Ron,
I am observing similar issue in my application. We are using Jboss 4.2.0.GA, Jboss
Remoting 2.5.3.SP1
We have a firewall between our server and client and we followed the forum,
http://blog.akquinet.de/2009/09/30/using-jboss-messaging-in-a-firewalled-...
http://blog.akquinet.de/2009/09/30/using-jboss-messaging-in-a-firewalled-...
to set our parameters like below,
+ <attribute name="marshaller"
isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>+
+ <attribute name="unmarshaller"
isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>+
+ <attribute name="dataType"
isParam="true">jms</attribute>+
+ <attribute name="socket.check_connection"
isParam="true">true</attribute> +
+ <attribute
name="serverBindAddress">${jboss.bind.address}</attribute>+
+ <attribute name="serverBindPort">4457</attribute>+
+ <attribute name="clientSocketClass"
isParam="true">org.jboss.jms.client.remoting.ClientSocketWrapper</attribute>+
+ <attribute
name="serverSocketClass">org.jboss.jms.server.remoting.ServerSocketWrapper</attribute>+
+ <attribute name="numberOfCallRetries"
isParam="true">5</attribute>+
+ <attribute name="pingFrequency"
isParam="true">5000</attribute>+
+ <attribute name="pingWindowFactor"
isParam="true">2</attribute>+
+ <attribute
name="onewayThreadPool">org.jboss.jms.server.remoting.DirectThreadPool</attribute> +
+ <attribute name="stopLeaseOnFailure"
isParam="true">true</attribute>+
+ <attribute name="clientLeasePeriod"
isParam="true">10000</attribute>+
+ <attribute name="validatorPingPeriod"
isParam="true">10000</attribute>+
+ <attribute name="validatorPingTimeout"
isParam="true">5000</attribute>+
+ <attribute name="failureDisconnectTimeout"
isParam="true">0</attribute>+
+ <attribute
name="callbackErrorsAllowed">1</attribute>+
+ <attribute
name="registerCallbackListener">false</attribute>+
+ <attribute name="useClientConnectionIdentity"
isParam="true">true</attribute>+
+ <attribute name="timeout"
isParam="true">1800000</attribute>+
+ <attribute name="numberOfRetries"
isParam="true">10</attribute>+
+ <attribute name="JBM_clientMaxPoolSize"
isParam="true">200</attribute>+
+ <attribute
name="callbackTimeout">10000</attribute>+
+ <attribute
name="secondaryBindPort">4481</attribute>+
With this implementation, we have mulitple Connections in CLOSE_WAIT state most of them
getting cleared and some left behind.
After several days, we loose the JMS communication. We need to restart the application
again to get rid off.
Also we observed the below exception multiple times when the issue happens,
Exception:
2011-03-28 11:50:10,271 TRACE
[org.jboss.remoting.transport.socket.ServerThread(638)](WorkerThread#4[10.255.254.134:42933])
WorkerThread#4[10.255.254.134:42933] EOFException received. This is likely due to client
finishing communication.
java.io.EOFException
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:740)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:586)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
Kindly share some light to resolve this.
Regards,
Joseph.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/596026#596026]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
3:38 p.m.
Ron Sigal [http://community.jboss.org/people/ron_sigal] created the discussion
"Re: Sockets left in CLOSE_WAIT on server for JMS connections"
To view the discussion, visit: http://community.jboss.org/message/601533#601533
--------------------------------------------------------------
Hi Joseph,
Hmmm. When ServerThread gets the EOFException, it should close the socket which should
move it out of the CLOSE_WAIT state. Can you identify the socket mentioned in the
exception (the one with remote port 42933 in this case) with a socket in the CLOSE_WAIT
state? Also, do you see the output of either
log.debug(this + " closing socketWrapper: " + socketWrapper);
or
log.error("failed to close socket wrapper", ex);
for that socket?
-Ron
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/601533#601533]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
11:24 a.m.
Joseph S [http://community.jboss.org/people/jselvin] created the discussion
"Re: Sockets left in CLOSE_WAIT on server for JMS connections"
To view the discussion, visit: http://community.jboss.org/message/604996#604996
--------------------------------------------------------------
Hi Ron,
We changed the pingFrequency to 180000 and updated the Remoting Jar to 2.5.4.
This issue seems not appearing nowadays :-)
But we are facing the issue in our client,
Our client Application is losing the connection sometimes(not consistently) and never
comes up,
We need to stop the client(but not server) to recreate the session successfully.
These are the below exceptions we see during that time.
+*[12 May 2011 16:52:36] WARN [Timer-3] (BisocketServerInvoker.java:1281) -
org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@6105a6:
detected failure on control connection Thread[control:
Socket[addr=omcr-rmg3.com/10.255.254.134,port=4481,localport=3132],6,]
(4ss2s-vevhkq-gnlg644p-1-gnlg6c36-l: requesting new control connection*+
+*[12 May 2011 16:52:36] DEBUG [controlConnectionRecreate:control:
Socket[addr=omcr-rmg3.com/10.255.254.134,port=4481,localport=3132]]
(BisocketClientInvoker.java:628) - getting secondary locator*+
+*[12 May 2011 16:52:36] DEBUG [controlConnectionRecreate:control:
Socket[addr=omcr-rmg3.com/10.255.254.134,port=4481,localport=3132]]
(BisocketClientInvoker.java:636) - secondary locator: InvokerLocator
[null://omcr-rmg3.com:4481/?]*+
+*[12 May 2011 16:52:36] DEBUG [controlConnectionRecreate:control:
Socket[addr=omcr-rmg3.com/10.255.254.134,port=4481,localport=3132]]
(BisocketServerInvoker.java:379) - creating control connection: InvokerLocator
[null://omcr-rmg3.com:4481/?]*+
+*[12 May 2011 16:52:36] DEBUG [controlConnectionRecreate:control:
Socket[addr=omcr-rmg3.com/10.255.254.134,port=4481,localport=3132]]
(BisocketServerInvoker.java:463) - SocketServerInvoker[172.28.2.28:345279834] created
control connection (4ss2s-vevhkq-gnlg644p-1-gnlg6c36-l):
Socket[addr=omcr-rmg3.com/10.255.254.134,port=4481,localport=3153]*+
Also attached the complete log file.
Kindly provide your analysis for this issue.
Thanks.
Joseph.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/604996#604996]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:22 a.m.
Ron Sigal [http://community.jboss.org/people/ron_sigal] created the discussion
"Re: Sockets left in CLOSE_WAIT on server for JMS connections"
To view the discussion, visit: http://community.jboss.org/message/622005#622005
--------------------------------------------------------------
Hi Joseph,
Are you still having this problem?
When you get a "+*detected failure on control connection"*+ message, it means a
PING from the server to the client isn't arriving in time. These failures might be a
sign of a congested network, in which case you might want to lengthen the PING cycle. You
can play with the parameters
* pingFrequency: how often the ping is sent
* pingWindowFactor: multiplied by pingFrequency to give the window within which the PING
is expected.
-Ron
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/622005#622005]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
4872
days inactive
5018
days old
3 comments
2 participants
participants (2)
-
Joseph S -
Ron Sigal