[forge-dev] Forge performance considerations / Bootstrap / modularity
Lincoln Baxter, III
lincolnbaxter at gmail.com
Mon Nov 5 11:45:05 EST 2012
Also, anything you can speed up with Jandex might also be very useful in
2.0! So let's definitely take a good look :)
On Mon, Nov 5, 2012 at 11:44 AM, Lincoln Baxter, III <
lincolnbaxter at gmail.com> wrote:
> Great!
>
> Please send a pull request for the ObserverCapture issue! Thanks!
>
> Good plan with the bootstrap API, and are you sure that Jandex would
> require changes in Weld-API? Look at what has been done in 2.0 with the
> https://github.com/forge/core/blob/2.0/plugin-container/src/main/java/org/jboss/forge/container/modules/ModularWeldDeployment.java
>
> What APIs did you have to change?
>
> Looking forward to seeing your changes!
>
> ~Lincoln
>
>
>
> On Mon, Nov 5, 2012 at 2:10 AM, Thomas Frühbeck <fruehbeck at aon.at> wrote:
>
>> The first issue (ObserverCaptureExtension.scan) is the easiest and most
>> improving, it only needs your approval that it's really up to the point -
>> it's just a few lines in fact.
>> I will provide a proposal in my clone at github and notify you.
>>
>> The Bootstrap redesign idea I will provide in a seperate branch in same
>> repo.
>> The Jandex-idea would need also changes in the Weld-API, so they are
>> somewhat theoretical :-/
>>
>> Regards,
>> Thomas
>>
>> Am 05.11.2012 01:27, schrieb Lincoln Baxter, III:
>>
>> I think performance improvements in Forge 1.x are very important even now!
>>
>> It's going to be around for a very long time and while we are working on
>> Forge 2.0 now, the kinds of improvements you are talking about, Thomas, are
>> very significant!
>>
>> How can we help? Do you think you could put this code into a branch with
>> a single commit (or just one for each change) so that we can see just how
>> much you had to change?
>>
>> The problem with the weld container is the Hibernate Tools plugin. If
>> that one works, and the Openshift and AS7 plugins work, then it's safe to
>> assume that it's working fine. It was a hack to solve a classloading
>> problem, but if we can fix it now, that's great!
>>
>> ~Lincoln
>>
>>
>> On Sat, Nov 3, 2012 at 3:27 PM, George Gastaldi <ggastald at redhat.com>wrote:
>>
>>> Hi Thomas,
>>>
>>> Nice to know that you've found some improvements for Forge 1.x startup
>>> time. However, we are rewriting Forge in 2.x from the ground up and we're
>>> focusing on performance and flexibility in first place.
>>>
>>> We encourage you to try the 2.0 branch and help us on finding some
>>> possible performance problems if you feel so.
>>>
>>> As for now, we are avoiding any major changes and updates to 1.x.
>>>
>>> Best Regards,
>>>
>>> George Gastaldi
>>>
>>> Em 03/11/2012, às 15:41, Thomas Frühbeck <fruehbeck at aon.at> escreveu:
>>>
>>> > Hi,
>>> >
>>> > following the discussions about boot-up performance of Forge I made
>>> some
>>> > investigations using a profiler.
>>> > After testing a lot of different scenarios I would like to share my
>>> > results and ask for your opinion on it.
>>> >
>>> >
>>> *************************************************************************************
>>> > Issue 1.: hot spot inv.
>>> > org.jboss.forge.bus.cdi.ObserverCaptureExtension.scan
>>> > profiling shows that 37% CPU time are used in firing
>>> > ProcessAnnotatedType events, which are AFAICS only used to filter old
>>> > event types
>>> > "37,2% - 13.442 ms - 1.866 inv.
>>> > org.jboss.weld.bootstrap.events.ProcessAnnotatedTypeImpl.fire"
>>> >
>>> > By introducing a preparatory check in ObserverCaptureExtension like
>>> >> if (isMethodsAnnotated(originalType, Observes.class)) {
>>> >> ....
>>> >> }
>>> >
>>> > and something like:
>>> >> private boolean isMethodsAnnotated(AnnotatedType<?> type, Class<?
>>> > extends Annotation> annotation) {
>>> >> if (type.getClass().isAnnotationPresent(annotation))
>>> >> return true;
>>> >> for (Method m : type.getClass().getMethods()) {
>>> >> if (m.isAnnotationPresent(annotation))
>>> >> return true;
>>> >> for (Annotation[] arr : m.getParameterAnnotations()) {
>>> >> for (Annotation a : arr) {
>>> >> if (a.equals(annotation))
>>> >> return true;
>>> >> }
>>> >> }
>>> >> }
>>> >> return false;
>>> >> }
>>> >
>>> > we could reduce startup by 20%
>>> > "13,8% - 2.438 ms - 1.798 inv.
>>> > org.jboss.weld.bootstrap.events.ProcessAnnotatedTypeImpl.fire
>>> >
>>> > Test: call Forge with commands "list-plugins" and "exit", like ~> time
>>> > ((echo forge list-plugins; echo exit;)| bin/forge)
>>> >
>>> > Before optimization:
>>> > ~/forge> time ((echo forge list-plugins && echo exit)| bin/forge )
>>> > _____
>>> > | ___|__ _ __ __ _ ___
>>> > | |_ / _ \| `__/ _` |/ _ \ \\
>>> > | _| (_) | | | (_| | __/ //
>>> > |_| \___/|_| \__, |\___|
>>> > |___/
>>> >
>>> > [no project] forge-distribution-1.1.1-SNAPSHOT $ forge list-plugins
>>> >
>>> org.richfaces.forge.richfaces-forge-plugin:1.0.5.Final:1.0.0-SNAPSHOT-b1a6ebed-462f-40e1-ac08-82f927489301
>>> >
>>> com.ocpsoft.forge.prettyfaces-plugin:1.0.2.Final:1.0.0-SNAPSHOT-c6b071ad-a6df-4112-8fff-82ecccd003fb
>>> >
>>> org.jboss.hibernate.forge.hibernate-tools-plugin:1.0.5.Final:1.0.0-SNAPSHOT-afb1da29-a99f-4dd7-b020-5f3ba6073cde
>>> >
>>> org.arquillian.forge.arquillian-plugin:1.0.3-SNAPSHOT:1.0.0-SNAPSHOT-42907982-d02e-4a1d-8ea5-8e66c8013fde
>>> > [no project] forge-distribution-1.1.1-SNAPSHOT $ exit
>>> >
>>> >
>>> > real 0m9.721s
>>> > user 0m26.533s
>>> > sys 0m0.433s
>>> >
>>> > After optimization:
>>> > ~forge1.1.1-weld1.2> time ((echo forge list-plugins; echo exit;)|
>>> bin/forge)
>>> > Using Forge at /raid/home/thomas/java/forge1.1.1-weld1.2
>>> > _____
>>> > | ___|__ _ __ __ _ ___
>>> > | |_ / _ \| `__/ _` |/ _ \ \\
>>> > | _| (_) | | | (_| | __/ //
>>> > |_| \___/|_| \__, |\___|
>>> > |___/
>>> >
>>> > [no project] forge1.1.1-weld1.2 $ forge list-plugins
>>> >
>>> org.richfaces.forge.richfaces-forge-plugin:1.0.5.Final:1.0.0-SNAPSHOT-b1a6ebed-462f-40e1-ac08-82f927489301
>>> >
>>> com.ocpsoft.forge.prettyfaces-plugin:1.0.2.Final:1.0.0-SNAPSHOT-c6b071ad-a6df-4112-8fff-82ecccd003fb
>>> >
>>> org.jboss.hibernate.forge.hibernate-tools-plugin:1.0.5.Final:1.0.0-SNAPSHOT-afb1da29-a99f-4dd7-b020-5f3ba6073cde
>>> >
>>> org.arquillian.forge.arquillian-plugin:1.0.3-SNAPSHOT:1.0.0-SNAPSHOT-42907982-d02e-4a1d-8ea5-8e66c8013fde
>>> > [no project] forge1.1.1-weld1.2 $ exit
>>> >
>>> >
>>> > real 0m6.016s
>>> > user 0m14.482s
>>> > sys 0m0.449s
>>> >
>>> >
>>> >
>>> *************************************************************************************
>>> > Issue 2.: double starting of Weld container
>>> > I admit that I do not fully understand, what I did and if this really
>>> > provides the full solution, but it worked and produced a fully
>>> > functional Forge instance.
>>> >
>>> > I tried to restructure the Bootstrap logic a bit:
>>> >
>>> > initializeClassloader();
>>> > WeldContainer container = weld.initialize();
>>> > manager = container.getBeanManager();
>>> >
>>> > try
>>> > {
>>> > loadPlugins();
>>> > weld.reInitialize();
>>> >
>>> > where initializeClassloader already uses the CompositeClassloader:
>>> >
>>> > private static void initializeClassloader() {
>>> > ModuleLoader moduleLoader = Module.getBootModuleLoader();
>>> >
>>> > CompositeClassLoader composite = new CompositeClassLoader();
>>> > composite.add(Module.forClassLoader(Bootstrap.class.getClassLoader(),
>>> > true).getClassLoader());
>>> >
>>> > Thread.currentThread().setContextClassLoader(composite);
>>> > }
>>> >
>>> > and loadPlugins simply adds more classloaders:
>>> >
>>> > CompositeClassLoader composite =
>>> > (CompositeClassLoader)Thread.currentThread().getContextClassLoader();
>>> > for (PluginEntry plugin : toLoad)
>>> > {
>>> > Module module =
>>> >
>>> moduleLoader.loadModule(ModuleIdentifier.fromString(plugin.toModuleId()));
>>> > composite.add(module.getClassLoader());
>>> >
>>> >
>>> > This way the classes are already available in the correct classloader
>>> > for the later ModularWeld.reInitialize():
>>> >
>>> > public void reInitialize() {
>>> > bootstrap.revisit();
>>> > bootstrap.validateBeans();
>>> > bootstrap.endInitialization();
>>> > }
>>> >
>>> > Where Bootstrap.revisit simply revisits the BeanDeployments which have
>>> > been newly found:
>>> >
>>> > private static class DeploymentVisitor {
>>> > .....
>>> > public Map<BeanDeploymentArchive, BeanDeployment> revisit() {
>>> > for (BeanDeploymentArchive archvive :
>>> > deployment.getBeanDeploymentArchives()) {
>>> > if
>>> > (!managerAwareBeanDeploymentArchives.containsKey(archvive)) {
>>> > visit(archvive, managerAwareBeanDeploymentArchives,
>>> > new HashSet<BeanDeploymentArchive>(), true);
>>> > BeanDeployment bd =
>>> > managerAwareBeanDeploymentArchives.get(archvive);
>>> > bd.createBeans(environment);
>>> > bd.deployBeans(environment);
>>> > }
>>> > }
>>> > return managerAwareBeanDeploymentArchives;
>>> > }
>>> >
>>> >
>>> >
>>> *************************************************************************************
>>> > Issue 3.: invocation of potentially expensive operations on
>>> > non-annotated or annotation-relevant classes
>>> > The current BeanDeployer simply invokes ProcessAnnotatedType on any
>>> > class, w/o checking, if this class is used by any annotation anywhere.
>>> > By jandexing all jars and assembling the dispersed Jandex indices
>>> during
>>> > URLScanning we could reduce the expensive WeldClass conversion.
>>> >
>>> > Original:
>>> > 11.108 ms - 2.024 inv.
>>> org.jboss.weld.bootstrap.BeanDeployer.addClass
>>> > Jandex-aware:
>>> > 4.721 ms - 990 inv. org.jboss.weld.bootstrap.BeanDeployer.addClass
>>> >
>>> > Possible solution:
>>> > - 1. URLHandler scans Jandex file locations
>>> >
>>> > protected void addToDiscovered(String name, URL url) {
>>> > ...
>>> > } else if (name.endsWith(JANDEX_IDX)) {
>>> > discoveredJandexIndexUrls.add(url);
>>> > }
>>> > }
>>> >
>>> > - 2. URLScanner assembles all found Jandex indices:
>>> >
>>> > List<Index> jandexIndexes = new ArrayList<Index>();
>>> > for (URL jandexUrl : handler.getDiscoveredJandexIndexUrls()) {
>>> > jandexIndexes.add(new
>>> > IndexReader(jandexUrl.openStream()).read());
>>> > }
>>> >
>>> > - 3. BeanDeployer does expensive WeldClass loading only if class is
>>> > mentioned in the Jandex indices
>>> > - 3.a. first collapse the Jandex-indices to a simple list of class
>>> > names:
>>> > for (Index jandexIndex : jandexIndexes) {
>>> > for (List<AnnotationInstance> aiList :
>>> > jandexIndex.getAnnotations().values()) {
>>> > for (AnnotationInstance ai : aiList) {
>>> > AnnotationTarget at = ai.target();
>>> > if (at instanceof ClassInfo) {
>>> > jandexAnnotated.add(((ClassInfo)at).name().toString());
>>> > } else if (at instanceof MethodInfo) {
>>> >
>>> jandexAnnotated.add(((MethodInfo)at).declaringClass().name().toString());
>>> > } else if (at instanceof MethodParameterInfo) {
>>> >
>>> jandexAnnotated.add(((MethodParameterInfo)at).method().declaringClass().name().toString());
>>> > } else if (at instanceof FieldInfo) {
>>> >
>>> jandexAnnotated.add(((FieldInfo)at).declaringClass().name().toString());
>>> > }
>>> > }
>>> > }
>>> > }
>>> >
>>> > - 3.b. query the aggregated list of Jandex-aware classes
>>> >
>>> > public BeanDeployer addClass(String className) {
>>> > Class<?> clazz = loadClass(className);
>>> > if (clazz != null && isBeanCandidate(clazz) &&
>>> > jandexContains(className)) {
>>> > WeldClass<?> weldClass = loadWeldClass(clazz);
>>> >
>>> >
>>> > Yet unsolved issue is the question, wheter an index is found in the
>>> > corresponding Archive or not, so not to enforce Jandexing _all_ jars
>>> for
>>> > Forge.
>>> > Jandexing is very easy though: jandex -m <jar>
>>> >
>>> > The abovementioned solutions have been implemented and produced a
>>> > functional and correctly working Forge instance.
>>> > AFAICS the solution for issue 2 could be a means to solve the reloading
>>> > of plugins too, but I didn't implement and verify this.
>>> >
>>> > The sources are not available online, because the current
>>> implementation
>>> > is highly fragile and not safe for simple usage - too many changes in
>>> > APIs, dependencies, etc.
>>> >
>>> > Comments welcome!
>>> >
>>> > Regards,
>>> > Thomas
>>> >
>>> > _______________________________________________
>>> > forge-dev mailing list
>>> > forge-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>>
>>> _______________________________________________
>>> forge-dev mailing list
>>> forge-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>>
>>
>>
>>
>> --
>> Lincoln Baxter, III
>> http://ocpsoft.org
>> "Simpler is better."
>>
>>
>> _______________________________________________
>> forge-dev mailing listforge-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev
>>
>>
>>
>> _______________________________________________
>> forge-dev mailing list
>> forge-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>
>>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.org
> "Simpler is better."
>
--
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/forge-dev/attachments/20121105/de2ef4b5/attachment-0001.html
More information about the forge-dev
mailing list