[rules-dev] Rules compilation and unnecessary attempts to getResourceAsStream()

[Leonardo Gomes]

Hi Guys,

First of all, congrats for the 5.3 release!

I was having a look at the recent improvements done to rules compilation,
mentioned here<http://docs.jboss.org/drools/release/5.3.0.Final/droolsjbpm-introduction-docs/html_single/index.html#d0e47>,
and I unfortunately couldn't really realize a 3 times faster compilation :(

After some profiling, my problem seems to come from
some unsuccessful attempts to getResourceAsStream done
at org.drools.commons.jci.compilers.EclipseJavaCompiler. Basically on line
218, to be able to know if something is a type or not, it will get
a supposedly class name and try to load it, if it cannot be loaded then it
will infer that what it thought was a class was actually a package.

Here goes a call stack, to give some context. In this particular case, the
string parameter was "com.sample.org", thus not a type.

EclipseJavaCompiler$2.findType(String) line: 220
> EclipseJavaCompiler$2.findType(char[], char[][]) line: 215
> LookupEnvironment.askForType(PackageBinding, char[]) line: 127
> PackageBinding.getTypeOrPackage(char[]) line: 183
> MethodScope(Scope).getTypeOrPackage(char[], int, boolean) line: 2592
> MethodScope(Scope).getPackage(char[][]) line: 2266
> QualifiedTypeReference.getTypeBinding(Scope) line: 69
> QualifiedTypeReference(TypeReference).internalResolveType(Scope) line: 130
> QualifiedTypeReference(TypeReference).resolveType(BlockScope, boolean)
> line: 197
> SourceTypeBinding.resolveTypesFor(MethodBinding) line: 1370
> SourceTypeBinding.methods() line: 1090
> SourceTypeBinding.faultInTypesForFieldsAndMethods() line: 593
> CompilationUnitScope.faultInTypes() line: 447
> Compiler.process(CompilationUnitDeclaration, int) line: 752
> Compiler.compile(ICompilationUnit[]) line: 464
> EclipseJavaCompiler.compile(String[], ResourceReader, ResourceStore,
> ClassLoader, JavaCompilerSettings) line: 358
> EclipseJavaCompiler(AbstractJavaCompiler).compile(String[],
> ResourceReader, ResourceStore, ClassLoader) line: 49
> JavaDialect.compileAll() line: 369
> DialectCompiletimeRegistry.compileAll() line: 53
> PackageRegistry.compileAll() line: 71
> PackageBuilder.compileAll() line: 869
> PackageBuilder.addPackage(PackageDescr) line: 826
> PackageBuilder.addPackageFromDrl(Resource) line: 404
> PackageBuilder.addKnowledgeResource(Resource, ResourceType,
> ResourceConfiguration) line: 586
> KnowledgeBuilderImpl.add(Resource, ResourceType) line: 37
> DroolsTest.readKnowledgeBase() line: 52
> DroolsTest.main(String[]) line: 25


On the same lines, method isPackage on line 285, will try to load a "class
name" and if it can't be found it will assume it was actually a package
name and return true.

Again, an illustrative call stack:

>
> CompositeClassLoader.getResourceAsStream(String) line: 128
> EclipseJavaCompiler$2.isPackage(String) line: 288
> EclipseJavaCompiler$2.isPackage(char[][], char[]) line: 318
> LookupEnvironment.isPackage(char[][], char[]) line: 1262
> PackageBinding.findPackage(char[]) line: 77
> PackageBinding.getTypeOrPackage(char[]) line: 196
> MethodScope(Scope).getTypeOrPackage(char[], int, boolean) line: 2592
> MethodScope(Scope).getPackage(char[][]) line: 2266
> QualifiedTypeReference.getTypeBinding(Scope) line: 69
> QualifiedTypeReference(TypeReference).internalResolveType(Scope) line: 130
> QualifiedTypeReference(TypeReference).resolveType(BlockScope, boolean)
> line: 197
> SourceTypeBinding.resolveTypesFor(MethodBinding) line: 1370
> SourceTypeBinding.methods() line: 1090
> SourceTypeBinding.faultInTypesForFieldsAndMethods() line: 593
> CompilationUnitScope.faultInTypes() line: 447
> Compiler.process(CompilationUnitDeclaration, int) line: 752


Turns out that getting a resource when it does not exist can be pretty
expensive if you're compiling rules inside a big application with a lot of
stuff on your classpath, and this approach ends up tying the compilation
time of rules to the classloaders' search space.

I was wondering if we could have a better way to distinguish types and
packages or at least a configurable way. The first things that come to my
mind would be caching, in the lines of what is done for class loading in
the org.drools.util.CompositeClassLoader. Or have two different
strategies/implementations for the findType and isPackage methods, one that
would assume that you respect java conventions and parse the string to say
if it's a package or a type and another that wouldn't assume anything and
use the getResourceAsStream as it's done today. Then, we could choose via
configuration which strategy to use.

Any other ideas? Any work already being done in this area?

Cheers,
Leo.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-dev/attachments/20111105/6fd52c1d/attachment-0001.html