We are having issues with a JBOSS Application server running with the following versions
jboss messaging 1.4.6
jboss remoting 2.5.2
primarily being used for messaging purposes.
The server often locks up and the only way to get the clients to connect again is to restart the server.
Looking at the stack trace ,multiple workerthreads are waiting on this following lock
- waiting on <0x1990f6eb> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock)
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock.acquire(WriterPreferenceReadWriteLock.java:163)
org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.routeInternal(MessagingPostOffice.java:2210)
org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.route(MessagingPostOffice.java:515)
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:777)
org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:399)
org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:87)
org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_7280680627620114891.java)
org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_7280680627620114891.java)
org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)
org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:897)
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:768)
- locked <0x69b3f1c0> (a org.jboss.remoting.transport.socket.ServerThread)
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:721)
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:548)
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
Attached is the settings from the remoting-bisocket-service.xml file
<!--
<attribute name="writeTimeout" isParam="true">30000</attribute>
-->
<!-- End immutable parameters -->
<attribute name="stopLeaseOnFailure" isParam="true">true</attribute>
<!-- Periodicity of client pings. Server window by default is twice this figure -->
<attribute name="clientLeasePeriod" isParam="true">10000</attribute>
<attribute name="validatorPingPeriod" isParam="true">10000</attribute>
<attribute name="validatorPingTimeout" isParam="true">5000</attribute>
<attribute name="failureDisconnectTimeout" isParam="true">0</attribute>
<attribute name="callbackErrorsAllowed">1</attribute>
<attribute name="registerCallbackListener">false</attribute>
<attribute name="useClientConnectionIdentity" isParam="true">true</attribute>
<attribute name="timeout" isParam="true">0</attribute>
<!-- Max Number of connections in client pool. This should be significantly higher than
the max number of sessions/consumers you expect -->
<attribute name="JBM_clientMaxPoolSize" isParam="true">1000</attribute>
<!-- The maximum time to wait before timing out on trying to write a message to socket for delivery -->
<attribute name="callbackTimeout">10000</attribute>
I am trying to find out if its a configuration issue or a code issue on our side ?
Is it possible that one particular client is having problems and that in turn causes the server to lock up ?
Please help.