Re: [jboss-dev-forums] [JBoss Microcontainer Development] - Classloading and caching changes
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] replied to the discussion
"Classloading and caching changes"
To view the discussion, visit: http://community.jboss.org/message/543114#543114
--------------------------------------------------------------
So, once we determined what we need to do, this is how it's impled atm.
A new CachedLoader interface was introduced
public interface CacheLoader extends Loader
{
/**
* Check the class cache.
*
* @param classLoader the reference classloader (possibly null)
* @param name the name of the class
* @param path the path of the class resource
* @param allExports whether to look at all exports
* @return the class if cached
*/
Class<?> checkClassCache(BaseClassLoader classLoader, String name, String path, boolean allExports);
}
where the BaseClassLoaderDomain does before and after checks
public Class<?> checkClassCache(BaseClassLoader classLoader, String name, String path, boolean allExports)
{
Class<?> result = checkCacheBefore(classLoader, name, path, allExports);
if (result != null)
return result;
result = checkCacheAfter(classLoader, name, path, allExports);
if (result != null)
return result;
result = checkClassCacheLocally(classLoader, name, path, allExports);
if (result != null)
return result;
return null;
}
where the actual impl lives in ClassLoaderDomain
protected Class<?> checkCacheBefore(BaseClassLoader classLoader, String name, String path, boolean allExports)
{
if (parent == null || parent instanceof CacheLoader == false)
return null;
ClassFilter filter = getParentPolicy().getBeforeFilter();
if (filter.matchesClassName(name))
{
CacheLoader loader = (CacheLoader) parent;
return loader.checkClassCache(classLoader, name, path, allExports);
}
return null;
}
/**
* Only check parent after if we already blacklisted this resource.
*/
@Override
protected Class<?> checkCacheAfter(BaseClassLoader classLoader, String name, String path, boolean allExports)
{
if (parent == null || parent instanceof CacheLoader == false || isBlackListedClass(path) == false)
return null;
ClassFilter filter = getParentPolicy().getAfterFilter();
if (filter.matchesClassName(name))
{
CacheLoader loader = (CacheLoader) parent;
return loader.checkClassCache(classLoader, name, path, allExports);
}
return null;
}
The 3 current CacheLoader impls are
* BaseClassLoaderDomain (we already saw its impl)
* BaseDelegateLoader (it just delegates to its domain)
* ClassLoaderToLoaderAdapter (see below)
public Class<?> checkClassCache(BaseClassLoader bcl, String name, String path, boolean allExports)
{
if (findLoadedClass == null)
return null;
final ClassLoader classLoader = getClassLoader();
try
{
return (Class<?>) findLoadedClass.invoke(classLoader, name);
}
catch (Exception e)
{
log.warn("Unexpected error retrieving found class " + name + " from classloader " + classLoader, e);
return null;
}
}
I have also hacked a simple debug-able mock in Deployers project -- where we can check different deployment types
* http://anonsvn.jboss.org/repos/jbossas/projects/jboss-deployers/trunk/dep... http://anonsvn.jboss.org/repos/jbossas/projects/jboss-deployers/trunk/dep...
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543114#543114]
Start a new discussion in JBoss Microcontainer Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
Re: [jboss-dev-forums] [JBoss Microcontainer Development] - Classloading and caching changes
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] replied to the discussion
"Classloading and caching changes"
To view the discussion, visit: http://community.jboss.org/message/543110#543110
--------------------------------------------------------------
> "the classloader used is in the parent domain"
> But this isn't always the case?
> Only for this piece:
>
> // Try the before attempt (e.g. from the parent)
> Loader loader = *null*;
> *if* (findInParent)
> loader = findBeforeLoader(name);
> *if* (loader != *null*)
> *return* loader;
>
>
>
>
> But this could be cached -- if the policy permits it?
>
>
>
> // Next use any requesting classloader, this will look at everything not just what it exports
> *if* (classLoader != *null*)
> {
> *if* (trace)
> log.trace(*this* + " trying to load " + name + " from requesting " + classLoader);
> *if* (classLoader.getResourceLocally(name) != *null*)
> *return* classLoader.getLoader();
> }
>
That's why it can't update the Cache in the place you suggested.
For a classloader inside the domain it can globally cache, otherwise it can't.
It should cache in the domain where the classloader lives.
What needs changing is to check that cache according to the parent delegation
rules outside the synchronization block.
if (matchesParentBefore)
checkParentCache();
checkOurCache();
The difficulty comes in that we would like to add
if (matchesParentAfter)
checkParentCache();
But we can't because the next logical step is to see if we can load from our domain.
This however would work:
if (isBlackListedInOurDomain && matchesParentAfter)
checkParentCache();
because we know we won't load it from our domain.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543110#543110]
Start a new discussion in JBoss Microcontainer Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
Re: [jboss-dev-forums] [JBoss Microcontainer Development] - Classloading and caching changes
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] replied to the discussion
"Classloading and caching changes"
To view the discussion, visit: http://community.jboss.org/message/543109#543109
--------------------------------------------------------------
> Whether it caches things is decided elsewhere, based on the ClassLoaderPolicy rules.
> This code is just a safety check in the event that the class changes
> (somebody redeployed a classloader) and the global cache hasn't been flushed properly.
>
> As I said on the other response, it can't cache the load in that "globalClassCache", because
> the classloader used is in the parent domain. It would need some complicated
> flushing rules to cache it in the child domain as well - not my preferred solution.
"the classloader used is in the parent domain"
But this isn't always the case?
Only for this piece:
// Try the before attempt (e.g. from the parent)
Loader loader = null;
if (findInParent)
loader = findBeforeLoader(name);
if (loader != null)
return loader;
But this could be cached -- if the policy permits it?
// Next use any requesting classloader, this will look at everything not just what it exports
if (classLoader != null)
{
if (trace)
log.trace(this + " trying to load " + name + " from requesting " + classLoader);
if (classLoader.getResourceLocally(name) != null)
return classLoader.getLoader();
}
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543109#543109]
Start a new discussion in JBoss Microcontainer Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
Re: [jboss-dev-forums] [JBoss Microcontainer Development] - Classloading and caching changes
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] replied to the discussion
"Classloading and caching changes"
To view the discussion, visit: http://community.jboss.org/message/543107#543107
--------------------------------------------------------------
First a few questions.
> What's the idea behind "useLoadClassForParent" flag?
> * So, instead of going into that magic synch code, we try the parent before?
The loadClassForParent is a backwards compatibility thing to work around
a few issues that the more efficient resource check causes in some edge cases when it delegates to the java classpath.
It shouldn't be using it when a Domain delegates to another Domain.
That was something we spotted and fixed in 2.2.x, see around line old/new 689/920
http://fisheye.jboss.org/browse/JBossAS/projects/jboss-cl/trunk/classload... http://fisheye.jboss.org/browse/JBossAS/projects/jboss-cl/trunk/classload...
> Why doesn't this code write into global cache?
> * I guess only certain Loaders are valid for caching? e.g. depending on the policy or "OSGi" rules
>
> (from BaseClassLoaderDomain)
>
> Loader loader = findLoader(classLoader, path, allExports, findInParent);
> if (loader != null)
> {
> Thread thread = Thread.currentThread();
> ClassLoadingTask task = new ClassLoadingTask(name, classLoader, thread);
> ClassLoaderManager.scheduleTask(task, loader, false);
> Class<?> result = ClassLoaderManager.process(thread, task);
> ClassCacheItem item = globalClassCache.get(path);
> if (item != null)
> item.clazz = result;
> return result;
> }
Whether it caches things is decided elsewhere, based on the ClassLoaderPolicy rules.
This code is just a safety check in the event that the class changes
(somebody redeployed a classloader) and the global cache hasn't been flushed properly.
As I said on the other response, it can't cache the load in that "globalClassCache", because
the classloader used is in the parent domain. It would need some complicated
flushing rules to cache it in the child domain as well - not my preferred solution.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543107#543107]
Start a new discussion in JBoss Microcontainer Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
[JBoss Microcontainer Development] - Classloading and caching changes
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] created the discussion
"Classloading and caching changes"
To view the discussion, visit: http://community.jboss.org/message/543106#543106
--------------------------------------------------------------
It all started as a performance issue.
> A web app is using xerces, trying to load its DocumentBuilder class a lot (really a lot).
>
> Changing "useLoadClassForParent" on default domain doesn't help a lot.
> And the class isn't cached in domain's globalClassCache.
>
> The problem is that that DB xerces class is loaded in this block (xerces is in JBOSS_HOME/lib), but it's never cached.
> Hence BaseClassLoader always goes into domain and its synch block.
>
> Couldn't we cache something more inside this block?
>
> Went over the forums these past days, to find anything on the topic, and I found this:
> * http://community.jboss.org/thread/127814?start=15&tstart=0 http://community.jboss.org/thread/127814?start=15&tstart=0
When you have one domain delegating to another domain, it doesn't check
the parent domain cache until after it has entered the synchronization block.
There are two possible fixes:
1) Introduce some extra logic into the domain cache checking that knows
about the parent delegation. That needs to be done carefully otherwise
it will check the wrong cache for the parent delegation rules.
i.e. it needs to differentiate classes loaded from the parent
before/after the class load in the domain.
2) Cache the result from the parent in the child domain.
This has the issues mentioned on the forum thread
i) it doesn't differentiate what is loaded before or after the load from
the target domain
ii) it complicates the flushing in the event that the parent domain gets
flushed, e.g. a classloader is removed from the parent domain
I think (1) would be the best way to optimize it, by adding
checkParentCacheBefore/After() logic into the current cache checking
that gets invoked outside the synchronization block.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543106#543106]
Start a new discussion in JBoss Microcontainer Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
Re: [jboss-dev-forums] [JBoss Microcontainer Development POJO Server] - Implementing a non-flat deployment for Weld Integration
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] replied to the discussion
"Implementing a non-flat deployment for Weld Integration"
To view the discussion, visit: http://community.jboss.org/message/543083#543083
--------------------------------------------------------------
> You are right. The point is that ClassPools contain the algorithm for finding Classes and other resources in the classpath. This means we need to take care of imports and exports and other complex stuff.
Thinking about it a bit, I'm not sure we actually need that complex strucutre in Classpools.
I don't see why simple domain mapping wouldn't work -- which would super speed up things.
I need to think about it a bit more, and have a more detailed look.
Or perhaps Kabir can do this, if we cannot get current Javassist based Reflect to behave faster as Introspection. :-)
> The only thing that I see is duplicate here is RegisterModuleCallback/DomainRegistry. We need to keep track of modules and to map those to ClassLoaders. I'm duplicating this stuff for now, but we should definetly review this in the future.
No, you don't need that registry module callback. It should all be possible to do in deployers.
And at the same time you are able to get access to other things via DeploymentUnit.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543083#543083]
Start a new discussion in JBoss Microcontainer Development POJO Server at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
Re: [jboss-dev-forums] [JBoss Microcontainer Development POJO Server] - Implementing a non-flat deployment for Weld Integration
by Flavia Rainone
Flavia Rainone [http://community.jboss.org/people/flavia.rainone%40jboss.com] replied to the discussion
"Implementing a non-flat deployment for Weld Integration"
To view the discussion, visit: http://community.jboss.org/message/543072#543072
--------------------------------------------------------------
> Ales Justin wrote:
>
> > So, Ales, how do you think this should be done? I think we have plenty of code in the ClassPools that do exactly that, mirror the ClassLoader structure in a way that allows us to find the classes and xml files in META-INF dirs. Maybe there is a way of reusing part of the ClassPool code to do this?
> It is similar in a way, but I don't see how you would be able to re-use it.
>
You are right. The point is that ClassPools contain the algorithm for finding Classes and other resources in the classpath. This means we need to take care of imports and exports and other complex stuff.
For WELDINT-1, all we need to do is to see the module hierarchy as you suggested, as that allows us to map the modules to BDAs and to yet see which BDAs are visible to a specific BDA.
The only thing that I see is duplicate here is RegisterModuleCallback/DomainRegistry. We need to keep track of modules and to map those to ClassLoaders. I'm duplicating this stuff for now, but we should definetly review this in the future.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543072#543072]
Start a new discussion in JBoss Microcontainer Development POJO Server at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months
[JBoss Microcontainer Development] - Wildcard support in Dynamic-imports
by Ales Justin
Ales Justin [http://community.jboss.org/people/alesj] created the discussion
"Wildcard support in Dynamic-imports"
To view the discussion, visit: http://community.jboss.org/message/543069#543069
--------------------------------------------------------------
I've implemented an initial version of wildcard support.
e.g. Dynamic-import: com.acme.foo.*
When creating lazy delegate in Module (which we do for all dynamic requirements),
we differentiate between wildcards and plain dynamic import.
if (pr.isWildcard())
{
ClassLoaderPolicyFactory factory = new WildcardClassLoaderPolicyFactory(domain, item);
return new WildcardDelegateLoader(factory, filter);
}
Our wildcard policy tracks potential matching modules via ModuleRegistry notion from ClassLoading instance.
Hence wildcard factory registers the policy as a listener.
public ClassLoaderPolicy createClassLoaderPolicy()
{
WildcardClassLoaderPolicy policy = new WildcardClassLoaderPolicy(domain, requirement, module);
ClassLoading classLoading = domain.getClassLoading();
classLoading.addModuleRegistry(policy); // so we know when to reset on module change
return policy;
}
What our wildcard policy does is actually delegation to other existing matching modules.
public URL getResource(String path)
{
Module cached = resourceCache.get(path);
if (cached != null)
return cached.getResource(path);
ClassFilter filter = requirement.toClassFilter();
if (filter.matchesResourcePath(path))
{
for (Module m : modules)
{
URL url = m.getResource(path);
if (url != null)
{
resourceCache.put(path, m);
used.add(m);
return url;
}
}
}
return null;
}
"modules" list gets populated via tracking, where we only include potential matching modules.
public void addModule(Module module)
{
Domain md = getDomain(module);
if (md != null && module.canResolve(requirement))
{
boolean isAncestor = (domain != md); // not the same domain, so it must be ancestor
synchronized (this)
{
if (isAncestor)
{
if (domain.isParentFirst())
{
modules.add(0, module);
parentsBefore++;
}
else
modules.add(module);
}
else
modules.add(parentsBefore, module);
}
reset();
}
}
Same on removal, where we also try to bounce our module if we see that some module we used went away.
We also remove ourselves from module listening when undeploying.
public void removeModule(Module module)
{
synchronized (this)
{
if (modules.remove(module))
{
Domain md = getDomain(module);
boolean isAncestor = (domain != md);
if (isAncestor && domain.isParentFirst())
parentsBefore--;
reset();
}
}
boolean sameModule = this.module == module;
// Unregister this policy as module listener
if (sameModule)
{
ClassLoading classLoading = domain.getClassLoading();
classLoading.removeModuleRegistry(this);
this.module = null;
}
// It's not us (we're already uninstalling) and we used this, let's bounce.
if (used.remove(module) && sameModule == false)
{
LifeCycle lifeCycle = this.module.getLifeCycle();
if (lifeCycle != null && module.isCascadeShutdown() == false)
{
try
{
lifeCycle.bounce();
}
catch (Exception e)
{
throw new IllegalArgumentException("Error bouncing module: " + this.module);
}
}
}
}
Then comes the tricky part. :-)
Since the policy is mostly just used to find the matching resource, where its underlying classloader should do the real loading.
But in our case where we delegate all of the things, this needed some hacking into existing code.
To find the right ClassLoader I hacked BaseDelegateLoader's getBaseClassLoader method to protected (don't see why it should really be pckg protected), and then overriden it in WildcardDelegateLoader.
protected BaseClassLoader getBaseClassLoader(String message, String context)
{
ClassLoaderPolicy policy = getPolicy();
if (policy instanceof WildcardClassLoaderPolicy == false)
throw new IllegalArgumentException("Can only handle wildcard policy: " + policy);
WildcardClassLoaderPolicy wclp = (WildcardClassLoaderPolicy) policy;
return wclp.getBaseClassLoader(context);
}
Which then delegates to the policy with additional context parameter -- which is really the path of the requested resource.
BaseClassLoader getBaseClassLoader(String context)
{
Module m = findModule(context);
if (m != null)
{
ClassLoader cl = ClassLoading.getClassLoaderForModule(m);
if (cl instanceof BaseClassLoader)
return BaseClassLoader.class.cast(cl);
}
return null;
}
But the changes don't stop there, as the ClassLoadingManager/Task also asume there is a single matching classloader.
So, I introduced a new protected method on BaseClassLoaderPolicy.
/**
* Get the classloader based on classloading task.
*
* Since ClassLoadingTask ctor is package protected
* this method cannot be easily abused, since the only
* code that can instantiate ClassLoadingTask is our ClassLoaderManager.
*
* @param task the classloading task info
* @return the classloader
*/
protected synchronized BaseClassLoader getClassLoader(ClassLoadingTask task)
{
return getClassLoader();
}
Which is then overriden in wildcard policy.
protected BaseClassLoader getClassLoader(ClassLoadingTask task)
{
if (task == null)
throw new IllegalArgumentException("Null task");
String path = ClassLoaderUtils.classNameToPath(task.getClassName());
return getBaseClassLoader(path);
}
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/543069#543069]
Start a new discussion in JBoss Microcontainer Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&cont...]
15 years, 11 months