[forge-dev] Forge performance considerations / Bootstrap / modularity
Lincoln Baxter, III
lincolnbaxter at gmail.com
Mon Nov 5 11:44:40 EST 2012
Great!
Please send a pull request for the ObserverCapture issue! Thanks!
Good plan with the bootstrap API, and are you sure that Jandex would
require changes in Weld-API? Look at what has been done in 2.0 with the
https://github.com/forge/core/blob/2.0/plugin-container/src/main/java/org/jboss/forge/container/modules/ModularWeldDeployment.java
What APIs did you have to change?
Looking forward to seeing your changes!
~Lincoln
On Mon, Nov 5, 2012 at 2:10 AM, Thomas Frühbeck <fruehbeck at aon.at> wrote:
> The first issue (ObserverCaptureExtension.scan) is the easiest and most
> improving, it only needs your approval that it's really up to the point -
> it's just a few lines in fact.
> I will provide a proposal in my clone at github and notify you.
>
> The Bootstrap redesign idea I will provide in a seperate branch in same
> repo.
> The Jandex-idea would need also changes in the Weld-API, so they are
> somewhat theoretical :-/
>
> Regards,
> Thomas
>
> Am 05.11.2012 01:27, schrieb Lincoln Baxter, III:
>
> I think performance improvements in Forge 1.x are very important even now!
>
> It's going to be around for a very long time and while we are working on
> Forge 2.0 now, the kinds of improvements you are talking about, Thomas, are
> very significant!
>
> How can we help? Do you think you could put this code into a branch with a
> single commit (or just one for each change) so that we can see just how
> much you had to change?
>
> The problem with the weld container is the Hibernate Tools plugin. If that
> one works, and the Openshift and AS7 plugins work, then it's safe to assume
> that it's working fine. It was a hack to solve a classloading problem, but
> if we can fix it now, that's great!
>
> ~Lincoln
>
>
> On Sat, Nov 3, 2012 at 3:27 PM, George Gastaldi <ggastald at redhat.com>wrote:
>
>> Hi Thomas,
>>
>> Nice to know that you've found some improvements for Forge 1.x startup
>> time. However, we are rewriting Forge in 2.x from the ground up and we're
>> focusing on performance and flexibility in first place.
>>
>> We encourage you to try the 2.0 branch and help us on finding some
>> possible performance problems if you feel so.
>>
>> As for now, we are avoiding any major changes and updates to 1.x.
>>
>> Best Regards,
>>
>> George Gastaldi
>>
>> Em 03/11/2012, às 15:41, Thomas Frühbeck <fruehbeck at aon.at> escreveu:
>>
>> > Hi,
>> >
>> > following the discussions about boot-up performance of Forge I made some
>> > investigations using a profiler.
>> > After testing a lot of different scenarios I would like to share my
>> > results and ask for your opinion on it.
>> >
>> >
>> *************************************************************************************
>> > Issue 1.: hot spot inv.
>> > org.jboss.forge.bus.cdi.ObserverCaptureExtension.scan
>> > profiling shows that 37% CPU time are used in firing
>> > ProcessAnnotatedType events, which are AFAICS only used to filter old
>> > event types
>> > "37,2% - 13.442 ms - 1.866 inv.
>> > org.jboss.weld.bootstrap.events.ProcessAnnotatedTypeImpl.fire"
>> >
>> > By introducing a preparatory check in ObserverCaptureExtension like
>> >> if (isMethodsAnnotated(originalType, Observes.class)) {
>> >> ....
>> >> }
>> >
>> > and something like:
>> >> private boolean isMethodsAnnotated(AnnotatedType<?> type, Class<?
>> > extends Annotation> annotation) {
>> >> if (type.getClass().isAnnotationPresent(annotation))
>> >> return true;
>> >> for (Method m : type.getClass().getMethods()) {
>> >> if (m.isAnnotationPresent(annotation))
>> >> return true;
>> >> for (Annotation[] arr : m.getParameterAnnotations()) {
>> >> for (Annotation a : arr) {
>> >> if (a.equals(annotation))
>> >> return true;
>> >> }
>> >> }
>> >> }
>> >> return false;
>> >> }
>> >
>> > we could reduce startup by 20%
>> > "13,8% - 2.438 ms - 1.798 inv.
>> > org.jboss.weld.bootstrap.events.ProcessAnnotatedTypeImpl.fire
>> >
>> > Test: call Forge with commands "list-plugins" and "exit", like ~> time
>> > ((echo forge list-plugins; echo exit;)| bin/forge)
>> >
>> > Before optimization:
>> > ~/forge> time ((echo forge list-plugins && echo exit)| bin/forge )
>> > _____
>> > | ___|__ _ __ __ _ ___
>> > | |_ / _ \| `__/ _` |/ _ \ \\
>> > | _| (_) | | | (_| | __/ //
>> > |_| \___/|_| \__, |\___|
>> > |___/
>> >
>> > [no project] forge-distribution-1.1.1-SNAPSHOT $ forge list-plugins
>> >
>> org.richfaces.forge.richfaces-forge-plugin:1.0.5.Final:1.0.0-SNAPSHOT-b1a6ebed-462f-40e1-ac08-82f927489301
>> >
>> com.ocpsoft.forge.prettyfaces-plugin:1.0.2.Final:1.0.0-SNAPSHOT-c6b071ad-a6df-4112-8fff-82ecccd003fb
>> >
>> org.jboss.hibernate.forge.hibernate-tools-plugin:1.0.5.Final:1.0.0-SNAPSHOT-afb1da29-a99f-4dd7-b020-5f3ba6073cde
>> >
>> org.arquillian.forge.arquillian-plugin:1.0.3-SNAPSHOT:1.0.0-SNAPSHOT-42907982-d02e-4a1d-8ea5-8e66c8013fde
>> > [no project] forge-distribution-1.1.1-SNAPSHOT $ exit
>> >
>> >
>> > real 0m9.721s
>> > user 0m26.533s
>> > sys 0m0.433s
>> >
>> > After optimization:
>> > ~forge1.1.1-weld1.2> time ((echo forge list-plugins; echo exit;)|
>> bin/forge)
>> > Using Forge at /raid/home/thomas/java/forge1.1.1-weld1.2
>> > _____
>> > | ___|__ _ __ __ _ ___
>> > | |_ / _ \| `__/ _` |/ _ \ \\
>> > | _| (_) | | | (_| | __/ //
>> > |_| \___/|_| \__, |\___|
>> > |___/
>> >
>> > [no project] forge1.1.1-weld1.2 $ forge list-plugins
>> >
>> org.richfaces.forge.richfaces-forge-plugin:1.0.5.Final:1.0.0-SNAPSHOT-b1a6ebed-462f-40e1-ac08-82f927489301
>> >
>> com.ocpsoft.forge.prettyfaces-plugin:1.0.2.Final:1.0.0-SNAPSHOT-c6b071ad-a6df-4112-8fff-82ecccd003fb
>> >
>> org.jboss.hibernate.forge.hibernate-tools-plugin:1.0.5.Final:1.0.0-SNAPSHOT-afb1da29-a99f-4dd7-b020-5f3ba6073cde
>> >
>> org.arquillian.forge.arquillian-plugin:1.0.3-SNAPSHOT:1.0.0-SNAPSHOT-42907982-d02e-4a1d-8ea5-8e66c8013fde
>> > [no project] forge1.1.1-weld1.2 $ exit
>> >
>> >
>> > real 0m6.016s
>> > user 0m14.482s
>> > sys 0m0.449s
>> >
>> >
>> >
>> *************************************************************************************
>> > Issue 2.: double starting of Weld container
>> > I admit that I do not fully understand, what I did and if this really
>> > provides the full solution, but it worked and produced a fully
>> > functional Forge instance.
>> >
>> > I tried to restructure the Bootstrap logic a bit:
>> >
>> > initializeClassloader();
>> > WeldContainer container = weld.initialize();
>> > manager = container.getBeanManager();
>> >
>> > try
>> > {
>> > loadPlugins();
>> > weld.reInitialize();
>> >
>> > where initializeClassloader already uses the CompositeClassloader:
>> >
>> > private static void initializeClassloader() {
>> > ModuleLoader moduleLoader = Module.getBootModuleLoader();
>> >
>> > CompositeClassLoader composite = new CompositeClassLoader();
>> > composite.add(Module.forClassLoader(Bootstrap.class.getClassLoader(),
>> > true).getClassLoader());
>> >
>> > Thread.currentThread().setContextClassLoader(composite);
>> > }
>> >
>> > and loadPlugins simply adds more classloaders:
>> >
>> > CompositeClassLoader composite =
>> > (CompositeClassLoader)Thread.currentThread().getContextClassLoader();
>> > for (PluginEntry plugin : toLoad)
>> > {
>> > Module module =
>> >
>> moduleLoader.loadModule(ModuleIdentifier.fromString(plugin.toModuleId()));
>> > composite.add(module.getClassLoader());
>> >
>> >
>> > This way the classes are already available in the correct classloader
>> > for the later ModularWeld.reInitialize():
>> >
>> > public void reInitialize() {
>> > bootstrap.revisit();
>> > bootstrap.validateBeans();
>> > bootstrap.endInitialization();
>> > }
>> >
>> > Where Bootstrap.revisit simply revisits the BeanDeployments which have
>> > been newly found:
>> >
>> > private static class DeploymentVisitor {
>> > .....
>> > public Map<BeanDeploymentArchive, BeanDeployment> revisit() {
>> > for (BeanDeploymentArchive archvive :
>> > deployment.getBeanDeploymentArchives()) {
>> > if
>> > (!managerAwareBeanDeploymentArchives.containsKey(archvive)) {
>> > visit(archvive, managerAwareBeanDeploymentArchives,
>> > new HashSet<BeanDeploymentArchive>(), true);
>> > BeanDeployment bd =
>> > managerAwareBeanDeploymentArchives.get(archvive);
>> > bd.createBeans(environment);
>> > bd.deployBeans(environment);
>> > }
>> > }
>> > return managerAwareBeanDeploymentArchives;
>> > }
>> >
>> >
>> >
>> *************************************************************************************
>> > Issue 3.: invocation of potentially expensive operations on
>> > non-annotated or annotation-relevant classes
>> > The current BeanDeployer simply invokes ProcessAnnotatedType on any
>> > class, w/o checking, if this class is used by any annotation anywhere.
>> > By jandexing all jars and assembling the dispersed Jandex indices during
>> > URLScanning we could reduce the expensive WeldClass conversion.
>> >
>> > Original:
>> > 11.108 ms - 2.024 inv.
>> org.jboss.weld.bootstrap.BeanDeployer.addClass
>> > Jandex-aware:
>> > 4.721 ms - 990 inv. org.jboss.weld.bootstrap.BeanDeployer.addClass
>> >
>> > Possible solution:
>> > - 1. URLHandler scans Jandex file locations
>> >
>> > protected void addToDiscovered(String name, URL url) {
>> > ...
>> > } else if (name.endsWith(JANDEX_IDX)) {
>> > discoveredJandexIndexUrls.add(url);
>> > }
>> > }
>> >
>> > - 2. URLScanner assembles all found Jandex indices:
>> >
>> > List<Index> jandexIndexes = new ArrayList<Index>();
>> > for (URL jandexUrl : handler.getDiscoveredJandexIndexUrls()) {
>> > jandexIndexes.add(new
>> > IndexReader(jandexUrl.openStream()).read());
>> > }
>> >
>> > - 3. BeanDeployer does expensive WeldClass loading only if class is
>> > mentioned in the Jandex indices
>> > - 3.a. first collapse the Jandex-indices to a simple list of class
>> > names:
>> > for (Index jandexIndex : jandexIndexes) {
>> > for (List<AnnotationInstance> aiList :
>> > jandexIndex.getAnnotations().values()) {
>> > for (AnnotationInstance ai : aiList) {
>> > AnnotationTarget at = ai.target();
>> > if (at instanceof ClassInfo) {
>> > jandexAnnotated.add(((ClassInfo)at).name().toString());
>> > } else if (at instanceof MethodInfo) {
>> >
>> jandexAnnotated.add(((MethodInfo)at).declaringClass().name().toString());
>> > } else if (at instanceof MethodParameterInfo) {
>> >
>> jandexAnnotated.add(((MethodParameterInfo)at).method().declaringClass().name().toString());
>> > } else if (at instanceof FieldInfo) {
>> > jandexAnnotated.add(((FieldInfo)at).declaringClass().name().toString());
>> > }
>> > }
>> > }
>> > }
>> >
>> > - 3.b. query the aggregated list of Jandex-aware classes
>> >
>> > public BeanDeployer addClass(String className) {
>> > Class<?> clazz = loadClass(className);
>> > if (clazz != null && isBeanCandidate(clazz) &&
>> > jandexContains(className)) {
>> > WeldClass<?> weldClass = loadWeldClass(clazz);
>> >
>> >
>> > Yet unsolved issue is the question, wheter an index is found in the
>> > corresponding Archive or not, so not to enforce Jandexing _all_ jars for
>> > Forge.
>> > Jandexing is very easy though: jandex -m <jar>
>> >
>> > The abovementioned solutions have been implemented and produced a
>> > functional and correctly working Forge instance.
>> > AFAICS the solution for issue 2 could be a means to solve the reloading
>> > of plugins too, but I didn't implement and verify this.
>> >
>> > The sources are not available online, because the current implementation
>> > is highly fragile and not safe for simple usage - too many changes in
>> > APIs, dependencies, etc.
>> >
>> > Comments welcome!
>> >
>> > Regards,
>> > Thomas
>> >
>> > _______________________________________________
>> > forge-dev mailing list
>> > forge-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>
>> _______________________________________________
>> forge-dev mailing list
>> forge-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.org
> "Simpler is better."
>
>
> _______________________________________________
> forge-dev mailing listforge-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev
>
>
>
> _______________________________________________
> forge-dev mailing list
> forge-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/forge-dev
>
>
--
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/forge-dev/attachments/20121105/115086d4/attachment-0001.html
More information about the forge-dev
mailing list