April 2009 - jboss-dev-forums - Jboss List Archives

[Design of POJO Server] - Caching of classes in BaseClassLoaderDomain

by bstansberry＠jboss.com

Apologies if this topic has been discussed/resolved elsewhere. TBH I'm sure it has. If so, please just tell me to go search. But in any case the implications are big enough that it seems worth another discussion. Dominik Pospisil is running some perf tests and is seeing a roughly 10x degradation in the # of ejb3 sfsb requests/sec a cluster can handle w/ 5.1.0.Beta1 vs EAP 4.3. A thread dump on one of the nodes shows 300 threads contending for a BaseClassLoader object's monitor. The contention is through the loadClass(String, boolean) method, synchronized(this) statement at line 411. I believe the key issue here is we are not caching classes in BaseClassLoaderDomain. Following stack trace from the thread that currently holds the monitor illustrates: | "WorkerThread#189[10.16.88.183:37775]" prio=10 tid=0x66c81000 nid=0x5645 runnable [0x5c184000..0x5c185eb0] | java.lang.Thread.State: RUNNABLE | at java.lang.Throwable.fillInStackTrace(Native Method) | - locked <0xe74f8600> (a java.security.PrivilegedActionException) | at java.lang.Throwable.<init>(Throwable.java:241) | at java.lang.Exception.<init>(Exception.java:77) | at java.security.PrivilegedActionException.<init>(PrivilegedActionException.java:48) | at java.security.AccessController.doPrivileged(Native Method) | at java.net.URLClassLoader.findClass(URLClassLoader.java:188) | at java.lang.ClassLoader.loadClass(ClassLoader.java:307) | - locked <0x756a3fe8> (a sun.misc.Launcher$AppClassLoader) | at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) | - locked <0x756a3fe8> (a sun.misc.Launcher$AppClassLoader) | at java.lang.ClassLoader.loadClass(ClassLoader.java:300) | - locked <0x757cc198> (a org.jboss.system.NoAnnotationURLClassLoader) | at java.lang.ClassLoader.loadClass(ClassLoader.java:252) | at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) | - locked <0x757cc198> (a org.jboss.system.NoAnnotationURLClassLoader) | at java.lang.Class.forName0(Native Method) | at java.lang.Class.forName(Class.java:247) | at org.jboss.classloader.plugins.loader.ClassLoaderToLoaderAdapter.loadClass(ClassLoaderToLoaderAdapter.java:172) | at org.jboss.classloader.spi.ClassLoaderDomain.loadClassFromParent(ClassLoaderDomain.java:352) | at org.jboss.classloader.spi.ClassLoaderDomain.loadClassBefore(ClassLoaderDomain.java:307) | at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:246) | at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1102) | at org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:772) | at org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:415) | - locked <0x79d9a068> (a org.jboss.classloader.spi.base.BaseClassLoader) | at java.lang.ClassLoader.loadClass(ClassLoader.java:252) | at org.jboss.ejb3.metadata.annotation.AnnotationRepositoryToMetaData.loadClass(AnnotationRepositoryToMetaData.java:216) | at org.jboss.ejb3.metadata.annotation.AnnotationRepositoryToMetaData.hasAnnotation(AnnotationRepositoryToMetaData.java:328) | at org.jboss.aop.Advisor.hasAnnotation(Advisor.java:889) | at org.jboss.aop.Advisor.hasAnnotation(Advisor.java:865) | at org.jboss.ejb3.interceptors.aop.LifecycleCallbacks.createLifecycleCallbackInterceptors(LifecycleCallbacks.java:101) | at org.jboss.ejb3.EJBContainer.invokeCallback(EJBContainer.java:1112) | at org.jboss.ejb3.stateful.StatefulContainer.invokePrePassivate(StatefulContainer.java:668) | at org.jboss.ejb3.stateful.StatefulBeanContext.preReplicate(StatefulBeanContext.java:544) | at org.jboss.ejb3.cache.tree.StatefulTreeCache.putInCache(StatefulTreeCache.java:510) | at org.jboss.ejb3.cache.tree.StatefulTreeCache.create(StatefulTreeCache.java:123) | at org.jboss.ejb3.stateful.StatefulContainer.createSession(StatefulContainer.java:388) | at org.jboss.ejb3.session.SessionContainer.createSession(SessionContainer.java:677) | at org.jboss.ejb3.proxy.factory.session.stateful.StatefulSessionProxyFactoryBase.getNewSessionId(StatefulSessionProxyFactoryBase.java:296) | at org.jboss.ejb3.proxy.factory.session.stateful.StatefulSessionProxyFactoryBase.createProxyBusiness(StatefulSessionProxyFactoryBase.java:160) | at sun.reflect.GeneratedMethodAccessor304.invoke(Unknown Source) | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) | at java.lang.reflect.Method.invoke(Method.java:597) | at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:121) | at org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82) | at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:908) | at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:742) | - locked <0x805023c8> (a org.jboss.remoting.transport.socket.ServerThread) | at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:695) | at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:549) | at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:230) Here we have an EJB3 SFSB deployment, no loader repository configured so it is using the default classloading domain. The call stack is trying to load class javax.ejb.PrePassivate, which wasn't initialized by the EJB3 deployment's classloader, so the deployment's BaseClassLoader is asking the BaseClassLoadingDomain for the class. The class has already been loaded, by whatever classloader loaded common/lib/jboss-javaee.jar. This is an extremely common call pattern. AFAICT to return the already loaded class in EAP 4.3 vs 5.x we have, in simplified form: EAP 4.3 a single get from a concurrent hash map: return UnifiedLoaderRepository3.classes.get(name); AS 5.x 1) Ask the parent for the class, which calls ClassLoader.loadClass() proceeding all the way to the bootstrap classloader, executing privileged blocks throwing CNFEs wrapped inside PrivilegedActionExceptions at each level and then catching and discarding them. The above stack trace catches a thread in the middle of creating a PrivilegedActionException. 2) Executing BaseClassLoaderDomain.findLoader logic to find the Loader that loaded the class. 3) Scheduling and processing ClassLoadingTask to get the class from the Loader. Yikes! If the 4.x approach is too much of a "big ball of mud" can't we at least find a way to skip step #1 if the class was already loaded from one of the loaders associated with the domain? In forum thread at http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4218813#4218813 pbmwg reports a similar problem, with a 66% performance improvement via eliminating just one source of calls to BaseClassLoader.loadClass(). View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226629#4226629 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226629

16 years, 8 months

1
0
0 / 0

[Other JBoss Development Design] - Re: default append setting to log/server.log is false

by dimitris＠jboss.org

For development a standard FileAppender (non-rollgin) with append=false suits better, yes. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226626#4226626 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226626

16 years, 8 months

1
0
0 / 0

[Design of AOP on JBoss (Aspects/JBoss)] - Advisor.hasAnnotation handling

by bstansberry＠jboss.com

Some EJB3 perf testing Dominik Pospisil is doing is showing major contention around a synchronized block in BaseClassLoader.loadClass(). I'll open a separate thread on Design of POJO Server re: reducing the cost of that sync block; this thread is about an issue in AOP that's increasing the amount of contention on the monitor guarding the block. In the perf test, many threads are contending on the monitor via this call stack, which will get executed once per request to the ejb: | "WorkerThread#298[10.16.88.182:54023]" prio=10 tid=0x6534d800 nid=0x56be waiting for monitor entry [0x598ad000..0x598adf30] | java.lang.Thread.State: BLOCKED (on object monitor) | at org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:411) | - waiting to lock <0x79d9a068> (a org.jboss.classloader.spi.base.BaseClassLoader) | at java.lang.ClassLoader.loadClass(ClassLoader.java:252) | at org.jboss.ejb3.metadata.annotation.AnnotationRepositoryToMetaData.loadClass(AnnotationRepositoryToMetaData.java:216) | at org.jboss.ejb3.metadata.annotation.AnnotationRepositoryToMetaData.hasAnnotation(AnnotationRepositoryToMetaData.java:328) | at org.jboss.aop.Advisor.hasAnnotation(Advisor.java:889) | at org.jboss.aop.Advisor.hasAnnotation(Advisor.java:865) | at org.jboss.ejb3.interceptors.aop.LifecycleCallbacks.createLifecycleCallbackInterceptors(LifecycleCallbacks.java:101) | at org.jboss.ejb3.EJBContainer.invokeCallback(EJBContainer.java:1112) | at org.jboss.ejb3.stateful.StatefulContainer.invokePrePassivate(StatefulContainer.java:668) | at org.jboss.ejb3.stateful.StatefulBeanContext.preReplicate(StatefulBeanContext.java:544) | at org.jboss.ejb3.cache.tree.StatefulTreeCache.putInCache(StatefulTreeCache.java:510) | at org.jboss.ejb3.cache.tree.StatefulTreeCache.replicate(StatefulTreeCache.java:278) | at org.jboss.ejb3.cache.StatefulReplicationInterceptor.invoke(StatefulReplicationInterceptor.java:114) If the org.jboss.ejb3.metadata.annotation.AnnotationRepositoryToMetaData.loadClass call could be avoided, the sync block would not be hit. Following from an email thread discusses how it's unfortunate that loadClass() is called at all: "Brian Stansberry" wrote : | The other thing is loading this class at all seems unfortunate. Pseudo call stack: | | 1) LifecycleCallbacks.createLifecycleCallbackInterceptors --> Advisor.hasAnnotation(.. PrePassivate.class) | | here we have the class | | 2) Advisor.hasAnnotation --> AnnotationRepositoryToMetaData.hasAnnotation(... PrePassivate.class.getName()) | | Advisor converts to String, probably because the default impl of AnnotationRepository caches things w/ String keys. But special class AnnotationRepositoryToMetaData does not... | | "Carlo de Wolf" wrote : | | No, because Advisor calls the wrong method on AnnotationRepository. | | | | Two lines down all exceptions are being ignored because of EJB3. Why is that? | | If the annotation class can't be found by the annotation repository then it probably should return false, not throw a fit. | | | 3) AnnotationRepositoryToMetaData.hasAnnotation() --> ClassLoader.loadClass(String) | | to get back the class we have a ref to a couple frames back. :( | So, question is, if Advisor.hasAnnotation(Method m, Class<? extends Annotation> annotation) is called, can the call to AnnotationRepository be to hasAnnotation(Member m, Class annotation) rather than hasAnnotation(Member m, String annotation) ? Secondary question is Carlo's point about exception handling. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226619#4226619 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226619

16 years, 8 months

1
0
0 / 0

[Design of POJO Server] - Re: HDScanner scanPeriod

by bstansberry＠jboss.com

Note that the farm/ folder is completely scanned every HDScanner run. This is because the contents of a ClusteredDeploymentRepository (e.g. farm/) need to be kept in sync regardless of whether the affected file triggers a redeploy. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226602#4226602 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226602

16 years, 8 months

1
0
0 / 0

[Design of JBoss Deployment Framework] - Re: Scoping of components while registering as MBeans

by jaikiran

Sure, https://jira.jboss.org/jira/browse/JBDEPLOY-184 View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226601#4226601 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226601

16 years, 8 months

1
0
0 / 0

[Design of JBoss Deployment Framework] - Re: Scoping of components while registering as MBeans

by alesj

Can you please create a Deployer JIRA and assign it to me? View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226593#4226593 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226593

16 years, 8 months

1
0
0 / 0

[Design of JBoss Deployment Framework] - Re: Scoping of components while registering as MBeans

by jaikiran

This piece in org.jboss.deployers.structure.spi.helpers.ComponentDeploymentContext public ObjectName getObjectName() | { | if (objectName == null) | { | String name = getName(); | name = name.replace("\"", """); | String temp = "jboss.deployment:id=\"" + name + "\",type=Component"; | try | { | objectName = new ObjectName(temp); | } | catch (MalformedObjectNameException e) | { | throw new RuntimeException("Error creating object name: " + temp, e); | } | } | return objectName; | } | View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226587#4226587 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226587

16 years, 8 months

1
0
0 / 0

[Design of JBoss Deployment Framework] - Scoping of components while registering as MBeans

by jaikiran

WRT https://jira.jboss.org/jira/browse/JBAS-6821, the MBeans corresponding to MC components are being registered in global scope. i.e. if two deployments with the same component name are deployed in the server then they will fail with InstanceAlreadyBoundException. Could the MBean name be changed to take into account the unit being deployed? i.e. scope it by including the unit name? View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226586#4226586 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226586

16 years, 8 months

1
0
0 / 0

[Design of Messaging on JBoss (Messaging/JBoss)] - Re: Changes in our JCA Adapter

by clebert.suconic＠jboss.com

Properties defined on the ra.xml, are being set on org.jboss.messaging.ra.JBMResourceAdapter. Which contains the resource-adapter global config. Properties defined on the @MDB ActionConfigs, are being set on org.jboss.messaging.ra.inflow.JBMActivationSpec, which contains the MDB Inbound configuration. Properties defined on tx-connection-factory, are being set on org.jboss.messaging.ra.JBMMCFProperties, which contains the Connection Outbound configuration. Currently, the connection is only defined on JBMResourceAdapter. Both the inbound and outbound are aways getting the connection from this global place on the RA. My proposal, is to add properties on both the Spec and JBMMCFProperties. If the user specify any connection property at JBMMCFProperties or JBMResourceAdapter I will ignore the global-config, giving the user the sensation of overriding the property. This is actually the same way it is done with other properties such as User, Password and destination-type. The ra.xml defines the default/global value and that could be changed at either MDB of tx-connection-factory. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226585#4226585 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226585

16 years, 8 months

1
0
0 / 0

[Design of POJO Server] - Re: HDScanner scanPeriod

by alesj

"bob.mcwhirter" wrote : | Right now, changes to config/**.yml doesn't seem to cause my rails apps to redeploy, but config/ is a metadata path according to the StructureMetaData in these. | Currently I only monitor *.xml files. See the filter property with StructureModificationChecker in profile.xml. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226582#4226582 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4226582

16 years, 8 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

jboss-dev-forums April 2009