10:34 a.m.
I have a few thoughts that might be of interest.
Firstly, I'd be interested to see when you are logging the class name
being loaded. If you are logging it in loadClass, you might not be
seeing the actual correct load order because that method is ultimately
recursive. To get an accurate picture of what order that classes are
actually defined - and thus what order you can load them in order to
prevent contention on per-class locks within the CL - you should log
immediately _after_ defineClass completes for each class.
Secondly, while debugging a resource iteration performance problem a
user was having with a large number of deployments, I discovered that
contention for the lock on JarFile and ZipFile was a primary cause. The
workaround I employed was to keep a RAM-based List of the files in the
JAR, which can be iterated over without touching the lock.
When we're preloading classes, we're definitely going to see this same
kind of contention come up, because there's only one lock per JarFile
instance so you can only ever read one entry at a time, thus preventing
any kind of useful concurrency on a per-module basis.
Exploding the files out of the JarFile could expose this contention and
therefore might be useful as a test - but it would also skew the results
a little because you have no decompression overhead, and creating the
separate file streams hypothetically might be somewhat more (or less)
expensive. I joked about resurrecting jzipfile (which I killed off
because it was something like 20% slower at decompressing entries than
Jar/ZipFile) but it might be worth considering having our own JAR
extractor at some point with a view towards concurrency gains. If we go
this route, we could go even further and create an optimized module
format, which is an idea I think we've looked at a little bit in the
past; there are a few avenues of exploration here which could be
interesting.
At some point we also need to see how jaotc might improve things. It
probably won't improve class loading time directly, but it might improve
the processes by which class loading is done because all the one-off
bits would be precompiled. Also it's worth exploring whether the jimage
format has contention issues like this.
On 05/14/2017 06:36 PM, Stuart Douglas wrote:
When JIRA was being screwy on Friday I used the time to investigate
an
idea I have had for a while about improving our boot time performance.
According to Yourkit the majority of our time is spent in class loading.
It seems very unlikely that we will be able to reduce the number of
classes we load on boot (or at the very least it would be a massive
amount of work) so I investigated a different approach.
I modified ModuleClassLoader to spit out the name and module of every
class that is loaded at boot time, and stored this in a properties file.
I then created a simple Service that starts immediately that uses two
threads to eagerly load every class on this list (I used two threads
because that seemed to work well on my laptop, I think
Runtime.availableProcessors()/4 is probably the best amount, but that
assumption would need to be tested on different hardware).
The idea behind this is that we know the classes will be used at some
point, and we generally do not fully utilise all CPU's during boot, so
we can use the unused CPU to pre load these classes so they are ready
when they are actually required.
Using this approach I saw the boot time for standalone.xml drop from
~2.9s to ~2.3s on my laptop. The (super hacky) code I used to perform
this test is at
https://github.com/wildfly/wildfly-core/compare/master...stuartwdouglas:b...
I think these initial results are encouraging, and it is a big enough
gain that I think it is worth investigating further.
Firstly it would be great if I could get others to try it out and see if
they see similar gains to boot time, it may be that the gain is very
system dependent.
Secondly if we do decide to do this there are two approach that we can
use that I can see:
1) A hard coded list of class names that we generate before a release
(basically what the hack already does), this is simplest, but does add a
little bit of additional work to the release process (although if it is
missed it would be no big deal, as ClassNotFoundException's would be
suppressed, and if a few classes are missing the performance impact is
negligible as long as the majority of the list is correct).
2) Generate the list dynamically on first boot, and store it in the temp
directory. This would require the addition of a hook into JBoss Modules
to generate the list, but is the approach I would prefer (as first boot
is always a bit slower anyway).
Thoughts?
Stuart
_______________________________________________
wildfly-dev mailing list
wildfly-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/wildfly-dev
--
- DML