[JBoss JIRA] Created: (JBMESSAGING-1210) Simple messaging test unable to drive CPU utilization better than 20%
by P Duffy (JIRA)
Simple messaging test unable to drive CPU utilization better than 20%
---------------------------------------------------------------------
Key: JBMESSAGING-1210
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1210
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: EAP 4.2 running JBoss Messaging 1.3. Solaris 10 running
on Sun T2000 server. Note that T2000 is able to run 32 threads
concurrently. We routinely run various tests to confirm AS is fully
utilizing available CPU.
Reporter: P Duffy
Assigned To: Tim Fox
After several months, we have been unable to resolve a JBM performance issue. Recent JBoss consultant visit has not helped.
We narrowed down our performance test to the following:
- several instances of a multi threaded test client sending a simple
ObjectMessage into a single AS instance.
- a single MDB is deployed in the AS instance. This MDB consumes the
message and throws it away (does nothing with it).
When this test is run, the AS instance indicates messages are backing up
in the Q, yet the clients are unable to drive the CPU past 20% utilization.
We've tried the following tuning adjustments:
(1) maxSession Annotation on MDB:
@ActivationConfigProperty(propertyName = "maxSession", propertyValue
= "300")
(2) maxSize in server/default/deploy/ejb3-interceptors-aop.xml, on
Message Driven Bean:
@org.jboss.annotation.ejb.PoolClass
(value=org.jboss.ejb3.StrictMaxPool.class, maxSize=300, timeout=10000)
(3) maxPoolSize and numAcceptThreads in
server/default/deploy/jboss-messaging.sar/remoting-service.xml
<attribute name="maxPoolSize" isParam="true">300</attribute>
<attribute name="numAcceptThreads" isParam="true">50</attribute>
(4) maxPoolSize in ThreadPool in /server/default/conf/jboss-service.xml
<attribute name="MaximumPoolSize">800</attribute>
Specific documentation re: JBM performance tuning is utterly lacking.
We have tried to piece together various shreds of configuration from
support cases, consultant comments, etc. to no avail.
We need SPECIFIC direction from JBoss re: how to properly configure
messaging to fully utilize available CPU. Else identify underlying bugs
and resolve same.
P.S. We have performed some examination of the JBM 1.3 code. With the
huge caveat that we are not expert re: JBM internals, we have identified
one location at which 100+ threads appear to be waiting on a single
threaded synchronization point. See below.
-----Original Message-----
From: Mickael Graham (mickaelg)
Sent: Monday, December 17, 2007 5:21 PM
To: Nick Culpepper (nculpepp)
Subject: Re: Thread dump for MDBEcho test
Yes. Heaps of threads are blocked on this:
at
org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServe
rInvocationHandler.java:105)
- waiting to lock <0xfffffffeb17129b8> (a java.lang.Class)
This is where the performance problem is. Looking at the JBoss
messaging code:
public Object invoke(InvocationRequest invocation) throws Throwable
{
if (trace) { log.trace("invoking " + invocation); }
synchronized (JMSServerInvocationHandler.class)
{
if (closed)
{
throw new MessagingJMSException("Cannot handle invocation
since messaging server is not active (it is either starting up or
shutting down)");
}
RequestSupport request =
(RequestSupport)invocation.getParameter();
if (request instanceof
ConnectionFactoryCreateConnectionDelegateRequest)
{
//Create connection request
ConnectionFactoryCreateConnectionDelegateRequest cReq =
(ConnectionFactoryCreateConnectionDelegateRequest)request;
String remotingSessionId = cReq.getRemotingSessionID();
ServerInvokerCallbackHandler callbackHandler = null;
synchronized(callbackHandlers)
{
callbackHandler =
(ServerInvokerCallbackHandler)callbackHandlers.get(remotingSessionId);
}
if (callbackHandler != null)
{
log.debug("found calllback handler for remoting session
" + Util.guidToString(remotingSessionId));
cReq.setCallbackHandler(callbackHandler);
}
else
{
throw new IllegalStateException("Cannot find callback
handler " +
"for session id " +
remotingSessionId);
}
}
return request.serverInvoke();
}
}
Basically this is for delivering I believe and that looks single
threaded. Perhaps once its delivered, it will not be single threaded.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[JBoss JIRA] Created: (JBMESSAGING-1233) Concurrancy contention getting ID's
by Mike Clark (JIRA)
Concurrancy contention getting ID's
-----------------------------------
Key: JBMESSAGING-1233
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1233
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.4.0.SP3
Environment: N/A
Reporter: Mike Clark
Assigned To: Tim Fox
The getID method in IDManager appears to use a blocksize of 1 when creating new ID's . This can create a contention bottleneck:
"WorkManager(2)-600" daemon prio=10 tid=0x0000000101fbccc0 nid=0x4e5 waiting for monitor entry [0xfffffffaf28fd000..0xfffffffaf28ff628]
at org.jboss.messaging.core.impl.IDManager.getID(IDManager.java:123)
- waiting to lock <0xfffffffef0e77f68> (a org.jboss.messaging.core.impl.IDManager)
at org.jboss.messaging.core.impl.tx.TransactionRepository.createTransaction(TransactionRepository.java:265)
at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendTransaction(ServerConnectionEndpoint.java:472)
at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.org$jboss$jms$server$endpoint$advised$ConnectionAdvised$sendTransaction$aop(ConnectionAdvised.java:101)
at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
at org.jboss.jms.server.container.SecurityAspect.handleSendTransaction(SecurityAspect.java:195)
at sun.reflect.GeneratedMethodAccessor235.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
at org.jboss.jms.server.endpoint.advised.ConnectionAdvised$sendTransaction_N3268650789275322226.invokeNext(ConnectionAdvised$sendTransaction_N3268650789275322226.java)
at org.jboss.jms.server.endpoint.advised.ConnectionAdvised.sendTransaction(ConnectionAdvised.java)
at org.jboss.jms.wireformat.ConnectionSendTransactionRequest.serverInvoke(ConnectionSendTransactionRequest.java:82)
at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:771)
at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
at org.jboss.remoting.Client.invoke(Client.java:1634)
at org.jboss.remoting.Client.invoke(Client.java:548)
at org.jboss.remoting.Client.invoke(Client.java:536)
at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:221)
at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:92)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeNext(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
at org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
at org.jboss.jms.tx.ResourceManager.commitLocal(ResourceManager.java:209)
at org.jboss.jms.client.container.SessionAspect.handleCommit(SessionAspect.java:557)
at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect26.invoke(SessionAspect26.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate$commit_8461082169793485964.invokeNext(ClientSessionDelegate$commit_8461082169793485964.java)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
at org.jboss.jms.client.delegate.ClientSessionDelegate$commit_8461082169793485964.invokeNext(ClientSessionDelegate$commit_8461082169793485964.java)
at org.jboss.jms.client.delegate.ClientSessionDelegate.commit(ClientSessionDelegate.java)
at org.jboss.jms.client.JBossSession.commit(JBossSession.java:164)
at org.jboss.resource.adapter.jms.inflow.JmsServerSession$LocalDemarcationStrategy.end(JmsServerSession.java:336)
at org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:248)
at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:204)
at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:275)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:748)
at java.lang.Thread.run(Thread.java:595)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[JBoss JIRA] Created: (JBCACHE-1301) Checking debug level can improve performance under heavy load
by Martin Landua (JIRA)
Checking debug level can improve performance under heavy load
-------------------------------------------------------------
Key: JBCACHE-1301
URL: http://jira.jboss.com/jira/browse/JBCACHE-1301
Project: JBoss Cache
Issue Type: Quality Risk
Security Level: Public (Everyone can see)
Affects Versions: 1.4.1.SP8
Environment: Any
Reporter: Martin Landua
Assigned To: Manik Surtani
In the source file ActivationInterceptor.java we experienced severe performance downgrade due to the fact that the debug level is not being checked. The two log statements in lines 152 and 156 result in a call to toString() for the DataNode object. Worst case this can create a huge string which is obsolete if debug is turned off in log4j. Since this occurs in a synchronized block, performance slows down in a multithreaded environment.
Please consider the following change (checking for debug level):
synchronized(this) {
if (fqn != null && nodeRemoved)
// If the node is being removed, just remove it and don't perform
// activation processing
loader.remove(fqn);
else if (fqn != null && cache.exists(fqn) && loader.exists(fqn)) {
// Remove the node from the cache loader if it exists in memory,
// its attributes have been initialized, its children have been loaded,
// AND it was found in the cache loader (nodeLoaded = true).
// Then notify the listeners that the node has been activated.
DataNode n = getNode(fqn); // don't load
// node not null and attributes have been loaded?
if (n != null && !n.containsKey(TreeCache.UNINITIALIZED)) {
if (n.hasChildren()) {
if (allInitialized(n)) {
if (log.isDebugEnabled()) log.debug("children all initialized");
remove(fqn);
}
} else if (loaderNoChildren(fqn)) {
if (log.isDebugEnabled()) log.debug("no children " + n);
remove(fqn);
}
}
}
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[JBoss JIRA] Created: (JBAS-5270) ClassLoader not unloaded on undeploy
by Arto Huusko (JIRA)
ClassLoader not unloaded on undeploy
------------------------------------
Key: JBAS-5270
URL: http://jira.jboss.com/jira/browse/JBAS-5270
Project: JBoss Application Server
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: ClassLoading
Affects Versions: JBossAS-4.2.1.GA
Environment: JDK 1.6.0_04 amd64
Reporter: Arto Huusko
Assigned To: Scott M Stark
When an EAR is undeployed, its ClassLoader and classes loaded by it are not unloaded, which leads to out of memory error in permanent generation on redeploy.
It appears that the ClassLoader becomes an orphaned ClassLoader (see http://blogs.sun.com/edwardchou/entry/find_orphaned_classloaders), and apparently JVM is unable to GC such ClassLoader instances (if it were, I don't understand the point of the blog about orphaned loaders).
My simple test ear contains one stateless session EJB (2.1), with a simple "echo" method (takes string, returns string). Jhat shows that the only reference to the ClassLoader instance comes from static field m3 in class $ProxyNN, where $ProxyNN is loaded by the same ClassLoader (in case it makes any difference, m3 points to the create method of the local home interface of the session).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[JBoss JIRA] Commented: (JBPORTAL-531) Import/Export of Portal Page and Portlet Database Settings
by Mike Jackson (JIRA)
[ http://jira.jboss.com/jira/browse/JBPORTAL-531?page=comments#action_12401147 ]
Mike Jackson commented on JBPORTAL-531:
---------------------------------------
For my situation I'd prefer not to use portlet-instances.xml files. I'd rather just have a way to export the entire setup of the portal which would include the pages, the pages & portlet instances, security attributes, object "placement", preferences, etc. from the somewhere in the admin side of the portal. Of course having the export without an import wouldn't be terribly useful, so I'd want that also. In my environement I have approximately 6 portals that I periodically need to sync up, so having something along these lines would be a great time savings for me.
The import function would need to be an overlay of the existing setup, which probably goes without saying.
The file format doesn't neet to be in XML, so long as the export file is human readable/editable it doesn't matter what the format is. As long as it isn't too cryptic and hard to maintain. My above mentioned portals aren't all quite the same, so I would need to be able to make changes to the export file for each environment prior to doing the imports.
> Import/Export of Portal Page and Portlet Database Settings
> ----------------------------------------------------------
>
> Key: JBPORTAL-531
> URL: http://jira.jboss.com/jira/browse/JBPORTAL-531
> Project: JBoss Portal
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: Portal Server
> Affects Versions: 2.2 Final, 2.6.2 Final
> Reporter: James Williams
> Fix For: 2.8 Final
>
>
> There is a need to export portal page and portlet setting changes that are made via the "management" portlet GUI interface, and then import those changes in another JBoss Portal instance.
> For example - If I want to migrate all of my QA portal page/portlet changes to Production today, I have to hard code those changes in the XML descriptors. A db export/import utility is preferred.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months