[JBoss Remoting] - Out of Memory Error with ConcurrentReaderHashMap on Jboss 5.1.0
by Anil Mathew
Anil Mathew [http://community.jboss.org/people/amathewjboss1] created the discussion
"Out of Memory Error with ConcurrentReaderHashMap on Jboss 5.1.0"
To view the discussion, visit: http://community.jboss.org/message/626575#626575
--------------------------------------------------------------
Hi,
We are running on JBoss 5.1 / JDK 1.6 on Linux servers. During very high load we are seeing Out of Memory Error and it seems like related to the JBoss remoting.
We took a heap dump and the Memory Analyzer shows the below instance is taking 25% of the memory:
One instance of *"org.jboss.mx.server.MBeanServerImpl"* loaded by *"org.jboss.classloader.spi.base.BaseClassLoader @ 0x415b2fc8"* occupies *65,055,032 (24.64%)* bytes. The memory is accumulated in one instance of *"EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap$Entry[]"* loaded by *"org.jboss.bootstrap.NoAnnotationURLClassLoader @ 0x42f8ba78"*.
The details which is pointing ot the JBoss remoting classes:
Class Name Shallow Heap Retained Heap
1. EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap$Entry[2048] Shallow Heap => 8,208 Retained Heap => 45,693,888
EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap Shallow Heap => 56 Retained Heap => 45,761,640
EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap$Entry Shallow Heap => 24 Retained Heap => 45,761,688
EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap$Entry[64] Shallow Heap => 272 Retained Heap => 64,584,776
EDU.oswego.cs.dl.util.concurrent.ConcurrentReaderHashMap Shallow Heap => 56 Retained Heap => 64,621,408
org.jboss.mx.server.registry.BasicMBeanRegistry Shallow Heap => 48 Retained Heap => 64,621,680
org.jboss.mx.server.MBeanServerImpl Shallow Heap => 24 Retained Heap => 65,055,032
com.arjuna.ats.internal.jbossatx.agent.LocalJBossAgentImpl
java.lang.Thread @ 0x42135bb8 Signal Dispatcher Native Stack, Thread
org.jboss.invocation.pooled.server.PooledInvokerHA
org.jboss.invocation.pooled.server.PooledInvoker
org.jboss.remoting.transport.bisocket.BisocketServerInvoker
org.jboss.remoting.transport.socket.SocketServerInvoker
Any thoughts on this please?
Thanks
Anil Mathew
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/626575#626575]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
12 years, 7 months
[JBoss Remoting] - Messaging server lock up issues
by vidhya Baskaran
vidhya Baskaran [http://community.jboss.org/people/vbs_jboss_accnt] created the discussion
"Messaging server lock up issues"
To view the discussion, visit: http://community.jboss.org/message/624162#624162
--------------------------------------------------------------
We are having issues with a JBOSS Application server running with the following versions
jboss messaging 1.4.6
jboss remoting 2.5.2
primarily being used for messaging purposes.
The server often locks up and the only way to get the clients to connect again is to restart the server.
Looking at the stack trace ,multiple workerthreads are waiting on this following lock
- waiting on <0x1990f6eb> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock)
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock.acquire(WriterPreferenceReadWriteLock.java:163)
org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.routeInternal(MessagingPostOffice.java:2210)
org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.route(MessagingPostOffice.java:515)
org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:777)
org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:399)
org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:87)
org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_7280680627620114891.java)
org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)
sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)
org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_7280680627620114891.java)
org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)
org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)
org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:897)
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:768)
- locked <0x69b3f1c0> (a org.jboss.remoting.transport.socket.ServerThread)
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:721)
org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:548)
org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
Attached is the settings from the remoting-bisocket-service.xml file
<!--
<attribute name="writeTimeout" isParam="true">30000</attribute>
-->
<!-- End immutable parameters -->
<attribute name="stopLeaseOnFailure" isParam="true">true</attribute>
<!-- Periodicity of client pings. Server window by default is twice this figure -->
<attribute name="clientLeasePeriod" isParam="true">10000</attribute>
<attribute name="validatorPingPeriod" isParam="true">10000</attribute>
<attribute name="validatorPingTimeout" isParam="true">5000</attribute>
<attribute name="failureDisconnectTimeout" isParam="true">0</attribute>
<attribute name="callbackErrorsAllowed">1</attribute>
<attribute name="registerCallbackListener">false</attribute>
<attribute name="useClientConnectionIdentity" isParam="true">true</attribute>
<attribute name="timeout" isParam="true">0</attribute>
<!-- Max Number of connections in client pool. This should be significantly higher than
the max number of sessions/consumers you expect -->
<attribute name="JBM_clientMaxPoolSize" isParam="true">1000</attribute>
<!-- The maximum time to wait before timing out on trying to write a message to socket for delivery -->
<attribute name="callbackTimeout">10000</attribute>
I am trying to find out if its a configuration issue or a code issue on our side ?
Is it possible that one particular client is having problems and that in turn causes the server to lock up ?
Please help.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/624162#624162]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
12 years, 7 months
[JBoss Remoting] - LeasePinger, BisocketServerInvoker, Jboss remoting, Messaging
by Srivisakh Rajasekar
Srivisakh Rajasekar [http://community.jboss.org/people/srivisakh_r] created the discussion
"LeasePinger, BisocketServerInvoker, Jboss remoting, Messaging"
To view the discussion, visit: http://community.jboss.org/message/621140#621140
--------------------------------------------------------------
Hi,
I am facing the same issue as mentioned in the thread. http://community.jboss.org/message/584787#584787 http://community.jboss.org/message/584787
I am using Jboss 5.1.0 with Jboss Messaging.I am receiving the JMS messages from the server for processing in the client(ATG running on Jboss 5.1.0).
If the client is up and running and the server stops in between , i am getting a message as
+13:47:52,328 WARN [LeasePinger] org.jboss.remoting.LeasePinger$LeaseTimerTask@f038e9 failed to ping to server: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:+
+ java.io.IOException: Connection reset by peer: socket write error+
But if the server restarts again, still the client(which is UP stiill) dosent reconnects, but throws the following Warning.
+13:47:57,359 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=35055,localport=4235],5,] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:48:47,360 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4264],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:49:17,360 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4279],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:49:47,360 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4305],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:50:17,360 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4325],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:50:47,360 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4337],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:51:17,361 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4350],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:51:47,361 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4415],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:52:02,361 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4419],5,] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:52:32,361 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4422],5,jboss] (a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0: requesting new control connection+
+13:53:02,377 WARN [BisocketServerInvoker] org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@137940f: detected failure on control connection Thread[control: Socket[ad+
+dr=sjoshiw5.ad.nike.com/10.192.227.59,port=20352,localport=4427],5,jboss]+
+13:53:02,393 WARN [BisocketServerInvoker] Control connection a5f4l2r-alztsh-grdwtj0n-1-grdww9ue-a0 has been recreated 10 times.+
+13:53:02,393 WARN [BisocketServerInvoker] Assuming it is a connection to an old server, and will not restart+
I downloaded the jboss-remoting.jar 2.2.4( http://www.jboss.org/jbossremoting/downloads.html http://www.jboss.org/jbossremoting/downloads.html) , but still it dosent help me. I saw a JIRA ticket opened on the same( https://issues.jboss.org/browse/JBREM-1144 https://issues.jboss.org/browse/JBREM-1144) and downloaded the jar from there too. Still i m facing this issue.
Kindly let me know on what i m missing with respect to the version or should i have to do any additional configuration to make it working.
Thanks
Sri.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/621140#621140]
Start a new discussion in JBoss Remoting at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
12 years, 7 months
[JBoss Tools] - Eclipse Project fails to publish(Indigo + Tools 3.2 + AS 6.0)
by Jeff Thompson
Jeff Thompson [http://community.jboss.org/people/javajd] created the discussion
"Eclipse Project fails to publish(Indigo + Tools 3.2 + AS 6.0)"
To view the discussion, visit: http://community.jboss.org/message/637597#637597
--------------------------------------------------------------
Under current company approved software I am allowed to run the following combination:
Eclipse Indigo
JBoss Tools 3.2
JBoss AS 6.0
I am behind a firewall and not able to access the update sites. I manually installed the Server Adapter by:
1. Downloading the JBoss Tools 3.2 Zip file from the approved software drive
2. In Eclipse, Install New Software, and pointed at the jar file
3. Under Web and Java EE Development selected JBossAS Tools
With a simple "hello world" html page bundled up in a .war file, I verified deployement via the dos command line and successfully accessed the page.
I then, in Eclipse, went to the Server page, and configured a JBoss AS Server pointing at the 6.0 Installation Directory with the "hello world" project added. I started a web page within Eclipse and was able to see the "hello world" page.
I then changed the verbage on the html page to "hello worlds", in the Server Tab in Eclipse [republish] was the status. I then STOPPED the running server. I right mouse clicked the "hello world" project under the Server and selected "Full Publish" and received the following:
----------------------------------------------------------------------------------------
An internal error occurred during: "Publishing to JBoss 6.0 Server...".
org/jboss/ide/eclipse/as/core/extnesions/events/ServerLog
-----------------------------------------------------------------------------------------
I have googled and clicked several sites for clues for the better part of 3 hours only to find one very slight reference that saving changes to a file does not guarantee an automatic republish which does not match what I am seeing.
Please provide guidance on anything else I can do to provide a working relationship between Eclipse Indigo + JBoss 6.0 for development purposes, i.e. being able to publish to the Server.
Thank you in advance for your time and consideration!
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/637597#637597]
Start a new discussion in JBoss Tools at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
12 years, 7 months