[wildfly-dev] Speeding up WildFly boot time

[Stuart Douglas]

On Mon, May 15, 2017 at 10:27 PM, Rostislav Svoboda <rsvoboda at redhat.com>
wrote:

> Hi.
>
> I can confirm I see improvements in boot time with your changes.
> My HW is Lenovo T440s with Fedora 25, Intel(R) Core(TM) i7-4600U CPU (Base
> Frequency 2.10 GHz, Max Turbo 3.30 GHz)
>
> I executed 50 iterations of start - stop sequence [1], before execution 5x
> start - stop for "warmup"
>
> With your changes
>        Min: 3116        Max: 3761       Average: 3247.640000
>
> Without:
>        Min: 3442        Max: 4081       Average: 3580.840000
>
>
> > 1) A hard coded list of class names that we generate before a release
>
> This will improve first boot impression, little bit harder for maintaining
> the list for the final build.
>
> Property files could be located inside properties directory of dedicated
> module (<resource-root path="properties"/>). Properties directory could
> contain property files for delivered profiles.
>
> Layered products or customer modifications could deliver own property file.
>  e.g. predefined property file for standalone-openshift.xml in EAP image
> in OpenShift environment, I think they boot the server just once and throw
> away the whole docker image when something changes.
>
>
> > 2) Generate the list dynamically on first boot, and store it in the temp
>
> This looks like the most elegant thing to do. Question is how it will slow
> down the initial boot. People care about first boot impression, some blog
> writers do the mistake too.
>

It will not actually slow down the initial boot (at least not in a
measurable way), but the first boot would not get the benefit of this
optimisation.

Stuart


> This would also block boot time improvements for use-cases when you start
> the server just once - e.g. Docker, OpenShift.
>
> Also the logic should take into account which profile is loaded - e.g
> standalone.xml vs. standalone-full-ha.xml
>
> Rostislav
>
> [1]
> rm wildfly-11.0.0.Beta1-SNAPSHOT-preload/standalone/log/server.log
> rm wildfly-11.0.0.Beta1-SNAPSHOT/standalone/log/server.log
>
> for i in {1..50}; do
>   echo $i
>   wildfly-11.0.0.Beta1-SNAPSHOT-preload/bin/standalone.sh 1>/dev/null
> 2>&1 &
>   sleep 8
>   wildfly-11.0.0.Beta1-SNAPSHOT-preload/bin/jboss-cli.sh -c :shutdown
> 1>/dev/null 2>&1
> done
> grep WFLYSRV0025 wildfly-11.0.0.Beta1-SNAPSHOT-
> preload/standalone/log/server.log | sed "s/.*\(....\)ms.*/\1/g"  | awk
> 'NR == 1 { max=$1; min=$1; sum=0 }
>    { if ($1>max) max=$1; if ($1<min) min=$1; sum+=$1;}
>    END {printf "Min: %d\tMax: %d\tAverage: %f\n", min, max, sum/NR}'
>
>
> for i in {1..50}; do
>   echo $i
>   wildfly-11.0.0.Beta1-SNAPSHOT/bin/standalone.sh 1>/dev/null 2>&1 &
>   sleep 8
>   wildfly-11.0.0.Beta1-SNAPSHOT/bin/jboss-cli.sh -c :shutdown 1>/dev/null
> 2>&1
> done
> grep WFLYSRV0025 wildfly-11.0.0.Beta1-SNAPSHOT/standalone/log/server.log
> | sed "s/.*\(....\)ms.*/\1/g"  | awk 'NR == 1 { max=$1; min=$1; sum=0 }
>    { if ($1>max) max=$1; if ($1<min) min=$1; sum+=$1;}
>    END {printf "Min: %d\tMax: %d\tAverage: %f\n", min, max, sum/NR}'
>
>
> ----- Original Message -----
> > When JIRA was being screwy on Friday I used the time to investigate an
> idea I
> > have had for a while about improving our boot time performance.
> According to
> > Yourkit the majority of our time is spent in class loading. It seems very
> > unlikely that we will be able to reduce the number of classes we load on
> > boot (or at the very least it would be a massive amount of work) so I
> > investigated a different approach.
> >
> > I modified ModuleClassLoader to spit out the name and module of every
> class
> > that is loaded at boot time, and stored this in a properties file. I then
> > created a simple Service that starts immediately that uses two threads to
> > eagerly load every class on this list (I used two threads because that
> > seemed to work well on my laptop, I think Runtime.availableProcessors()/4
> is
> > probably the best amount, but that assumption would need to be tested on
> > different hardware).
> >
> > The idea behind this is that we know the classes will be used at some
> point,
> > and we generally do not fully utilise all CPU's during boot, so we can
> use
> > the unused CPU to pre load these classes so they are ready when they are
> > actually required.
> >
> > Using this approach I saw the boot time for standalone.xml drop from
> ~2.9s to
> > ~2.3s on my laptop. The (super hacky) code I used to perform this test
> is at
> > https://github.com/wildfly/wildfly-core/compare/master...
> stuartwdouglas:boot-performance-hack
> >
> > I think these initial results are encouraging, and it is a big enough
> gain
> > that I think it is worth investigating further.
> >
> > Firstly it would be great if I could get others to try it out and see if
> they
> > see similar gains to boot time, it may be that the gain is very system
> > dependent.
> >
> > Secondly if we do decide to do this there are two approach that we can
> use
> > that I can see:
> >
> > 1) A hard coded list of class names that we generate before a release
> > (basically what the hack already does), this is simplest, but does add a
> > little bit of additional work to the release process (although if it is
> > missed it would be no big deal, as ClassNotFoundException's would be
> > suppressed, and if a few classes are missing the performance impact is
> > negligible as long as the majority of the list is correct).
> >
> > 2) Generate the list dynamically on first boot, and store it in the temp
> > directory. This would require the addition of a hook into JBoss Modules
> to
> > generate the list, but is the approach I would prefer (as first boot is
> > always a bit slower anyway).
> >
> > Thoughts?
> >
> > Stuart
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170516/275a3653/attachment.html