12:40 p.m.
Hi,
following the discussions about boot-up performance of Forge I made some
investigations using a profiler.
After testing a lot of different scenarios I would like to share my
results and ask for your opinion on it.
*************************************************************************************
Issue 1.: hot spot inv.
org.jboss.forge.bus.cdi.ObserverCaptureExtension.scan
profiling shows that 37% CPU time are used in firing
ProcessAnnotatedType events, which are AFAICS only used to filter old
event types
"37,2% - 13.442 ms - 1.866 inv.
org.jboss.weld.bootstrap.events.ProcessAnnotatedTypeImpl.fire"
By introducing a preparatory check in ObserverCaptureExtension like
if (isMethodsAnnotated(originalType, Observes.class)) {
....
}
and something like:
private boolean isMethodsAnnotated(AnnotatedType<?> type,
Class<?
extends Annotation> annotation) {
if (type.getClass().isAnnotationPresent(annotation))
return true;
for (Method m : type.getClass().getMethods()) {
if (m.isAnnotationPresent(annotation))
return true;
for (Annotation[] arr : m.getParameterAnnotations()) {
for (Annotation a : arr) {
if (a.equals(annotation))
return true;
}
}
}
return false;
}
we could reduce startup by 20%
"13,8% - 2.438 ms - 1.798 inv.
org.jboss.weld.bootstrap.events.ProcessAnnotatedTypeImpl.fire
Test: call Forge with commands "list-plugins" and "exit", like ~>
time
((echo forge list-plugins; echo exit;)| bin/forge)
Before optimization:
~/forge> time ((echo forge list-plugins && echo exit)| bin/forge )
_____
| ___|__ _ __ __ _ ___
| |_ / _ \| `__/ _` |/ _ \ \\
| _| (_) | | | (_| | __/ //
|_| \___/|_| \__, |\___|
|___/
[no project] forge-distribution-1.1.1-SNAPSHOT $ forge list-plugins
org.richfaces.forge.richfaces-forge-plugin:1.0.5.Final:1.0.0-SNAPSHOT-b1a6ebed-462f-40e1-ac08-82f927489301
com.ocpsoft.forge.prettyfaces-plugin:1.0.2.Final:1.0.0-SNAPSHOT-c6b071ad-a6df-4112-8fff-82ecccd003fb
org.jboss.hibernate.forge.hibernate-tools-plugin:1.0.5.Final:1.0.0-SNAPSHOT-afb1da29-a99f-4dd7-b020-5f3ba6073cde
org.arquillian.forge.arquillian-plugin:1.0.3-SNAPSHOT:1.0.0-SNAPSHOT-42907982-d02e-4a1d-8ea5-8e66c8013fde
[no project] forge-distribution-1.1.1-SNAPSHOT $ exit
real 0m9.721s
user 0m26.533s
sys 0m0.433s
After optimization:
~forge1.1.1-weld1.2> time ((echo forge list-plugins; echo exit;)| bin/forge)
Using Forge at /raid/home/thomas/java/forge1.1.1-weld1.2
_____
| ___|__ _ __ __ _ ___
| |_ / _ \| `__/ _` |/ _ \ \\
| _| (_) | | | (_| | __/ //
|_| \___/|_| \__, |\___|
|___/
[no project] forge1.1.1-weld1.2 $ forge list-plugins
org.richfaces.forge.richfaces-forge-plugin:1.0.5.Final:1.0.0-SNAPSHOT-b1a6ebed-462f-40e1-ac08-82f927489301
com.ocpsoft.forge.prettyfaces-plugin:1.0.2.Final:1.0.0-SNAPSHOT-c6b071ad-a6df-4112-8fff-82ecccd003fb
org.jboss.hibernate.forge.hibernate-tools-plugin:1.0.5.Final:1.0.0-SNAPSHOT-afb1da29-a99f-4dd7-b020-5f3ba6073cde
org.arquillian.forge.arquillian-plugin:1.0.3-SNAPSHOT:1.0.0-SNAPSHOT-42907982-d02e-4a1d-8ea5-8e66c8013fde
[no project] forge1.1.1-weld1.2 $ exit
real 0m6.016s
user 0m14.482s
sys 0m0.449s
*************************************************************************************
Issue 2.: double starting of Weld container
I admit that I do not fully understand, what I did and if this really
provides the full solution, but it worked and produced a fully
functional Forge instance.
I tried to restructure the Bootstrap logic a bit:
initializeClassloader();
WeldContainer container = weld.initialize();
manager = container.getBeanManager();
try
{
loadPlugins();
weld.reInitialize();
where initializeClassloader already uses the CompositeClassloader:
private static void initializeClassloader() {
ModuleLoader moduleLoader = Module.getBootModuleLoader();
CompositeClassLoader composite = new CompositeClassLoader();
composite.add(Module.forClassLoader(Bootstrap.class.getClassLoader(),
true).getClassLoader());
Thread.currentThread().setContextClassLoader(composite);
}
and loadPlugins simply adds more classloaders:
CompositeClassLoader composite =
(CompositeClassLoader)Thread.currentThread().getContextClassLoader();
for (PluginEntry plugin : toLoad)
{
Module module =
moduleLoader.loadModule(ModuleIdentifier.fromString(plugin.toModuleId()));
composite.add(module.getClassLoader());
This way the classes are already available in the correct classloader
for the later ModularWeld.reInitialize():
public void reInitialize() {
bootstrap.revisit();
bootstrap.validateBeans();
bootstrap.endInitialization();
}
Where Bootstrap.revisit simply revisits the BeanDeployments which have
been newly found:
private static class DeploymentVisitor {
.....
public Map<BeanDeploymentArchive, BeanDeployment> revisit() {
for (BeanDeploymentArchive archvive :
deployment.getBeanDeploymentArchives()) {
if
(!managerAwareBeanDeploymentArchives.containsKey(archvive)) {
visit(archvive, managerAwareBeanDeploymentArchives,
new HashSet<BeanDeploymentArchive>(), true);
BeanDeployment bd =
managerAwareBeanDeploymentArchives.get(archvive);
bd.createBeans(environment);
bd.deployBeans(environment);
}
}
return managerAwareBeanDeploymentArchives;
}
*************************************************************************************
Issue 3.: invocation of potentially expensive operations on
non-annotated or annotation-relevant classes
The current BeanDeployer simply invokes ProcessAnnotatedType on any
class, w/o checking, if this class is used by any annotation anywhere.
By jandexing all jars and assembling the dispersed Jandex indices during
URLScanning we could reduce the expensive WeldClass conversion.
Original:
11.108 ms - 2.024 inv. org.jboss.weld.bootstrap.BeanDeployer.addClass
Jandex-aware:
4.721 ms - 990 inv. org.jboss.weld.bootstrap.BeanDeployer.addClass
Possible solution:
- 1. URLHandler scans Jandex file locations
protected void addToDiscovered(String name, URL url) {
...
} else if (name.endsWith(JANDEX_IDX)) {
discoveredJandexIndexUrls.add(url);
}
}
- 2. URLScanner assembles all found Jandex indices:
List<Index> jandexIndexes = new ArrayList<Index>();
for (URL jandexUrl : handler.getDiscoveredJandexIndexUrls()) {
jandexIndexes.add(new
IndexReader(jandexUrl.openStream()).read());
}
- 3. BeanDeployer does expensive WeldClass loading only if class is
mentioned in the Jandex indices
- 3.a. first collapse the Jandex-indices to a simple list of class
names:
for (Index jandexIndex : jandexIndexes) {
for (List<AnnotationInstance> aiList :
jandexIndex.getAnnotations().values()) {
for (AnnotationInstance ai : aiList) {
AnnotationTarget at = ai.target();
if (at instanceof ClassInfo) {
jandexAnnotated.add(((ClassInfo)at).name().toString());
} else if (at instanceof MethodInfo) {
jandexAnnotated.add(((MethodInfo)at).declaringClass().name().toString());
} else if (at instanceof MethodParameterInfo) {
jandexAnnotated.add(((MethodParameterInfo)at).method().declaringClass().name().toString());
} else if (at instanceof FieldInfo) {
jandexAnnotated.add(((FieldInfo)at).declaringClass().name().toString());
}
}
}
}
- 3.b. query the aggregated list of Jandex-aware classes
public BeanDeployer addClass(String className) {
Class<?> clazz = loadClass(className);
if (clazz != null && isBeanCandidate(clazz) &&
jandexContains(className)) {
WeldClass<?> weldClass = loadWeldClass(clazz);
Yet unsolved issue is the question, wheter an index is found in the
corresponding Archive or not, so not to enforce Jandexing _all_ jars for
Forge.
Jandexing is very easy though: jandex -m <jar>
The abovementioned solutions have been implemented and produced a
functional and correctly working Forge instance.
AFAICS the solution for issue 2 could be a means to solve the reloading
of plugins too, but I didn't implement and verify this.
The sources are not available online, because the current implementation
is highly fragile and not safe for simple usage - too many changes in
APIs, dependencies, etc.
Comments welcome!
Regards,
Thomas