[rules-users] Drools compiler memory & stateless sessions?

Kris Nuttycombe kris.nuttycombe at gmail.com
Wed Nov 26 14:41:14 EST 2008 

Thanks, Amit, I'll give that approach a try.

A bit of additional information that seems like it might be related: I find
that when I shut down the application server (Glassfish) in which this is
running, or unload the EAR, I see a bunch of errors like this in the logs:

[#|2008-11-26T19:29:34.240+0000|WARNING|sun-appserver9.1|javax.enterprise.system.core.classloading|_ThreadID=18;_ThreadName=RMI
TCP Connection(47)-172.20.10.129
;_RequestID=ac0e52f1-a56e-438a-acb3-a66a93c0e19
f;|Input stream has been finalized or forced closed without being explicitly
closed; stream instantiation reported in following stack trace
java.lang.Throwable
        at
com.sun.enterprise.loader.EJBClassLoader$SentinelInputStream.<init>(EJBClassLoader.java:1169)
        at
com.sun.enterprise.loader.EJBClassLoader$InternalJarURLConnection.getInputStream(EJBClassLoader.java:1262)
        at java.net.URL.openStream(URL.java:1009)
        at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1161)
        at
com.sun.enterprise.loader.EJBClassLoader.getResourceAsStream(EJBClassLoader.java:799)
        at
org.drools.rule.PackageCompilationData$PackageClassLoader.getResourceAsStream(PackageCompilationData.java:384)
        at
org.drools.commons.jci.compilers.EclipseJavaCompiler$2.isPackage(EclipseJavaCompiler.java:280)
        at
org.drools.commons.jci.compilers.EclipseJavaCompiler$2.findType(EclipseJavaCompiler.java:222)
        at
org.drools.commons.jci.compilers.EclipseJavaCompiler$2.findType(EclipseJavaCompiler.java:204)
        at
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:97)
        at
org.eclipse.jdt.internal.compiler.lookup.UnresolvedReferenceBinding.resolve(UnresolvedReferenceBinding.java:43)
        at
org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.resolveType(BinaryTypeBinding.java:97)
        at
org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:167)
        at
org.eclipse.jdt.internal.compiler.lookup.Scope.getPackage(Scope.java:2046)
        at
org.eclipse.jdt.internal.compiler.ast.QualifiedTypeReference.getTypeBinding(QualifiedTypeReference.java:69)
        at
org.eclipse.jdt.internal.compiler.ast.TypeReference.resolveType(TypeReference.java:130)
        at
org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.resolveTypesFor(SourceTypeBinding.java:1358)
        at
org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.methods(SourceTypeBinding.java:1099)
        at
org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.faultInTypesForFieldsAndMethods(SourceTypeBinding.java:583)
        at
org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInTypes(CompilationUnitScope.java:428)
        at
org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:589)
        at
org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:411)
        at
org.drools.commons.jci.compilers.EclipseJavaCompiler.compile(EclipseJavaCompiler.java:351)
        at
org.drools.commons.jci.compilers.AbstractJavaCompiler.compile(AbstractJavaCompiler.java:51)
        at
org.drools.rule.builder.dialect.java.JavaDialect.compileAll(JavaDialect.java:342)
        at
org.drools.compiler.DialectRegistry.compileAll(DialectRegistry.java:60)
        at
org.drools.compiler.PackageBuilder.addPackage(PackageBuilder.java:308)
        at
org.drools.compiler.PackageBuilder.addPackageFromDrl(PackageBuilder.java:167)

Why is the stream of the compiled class not being closed? Could this be
related?

Thanks,

Kris

On Wed, Nov 26, 2008 at 10:51 AM, Amit Mhatre <ajmhatre at gmail.com> wrote:

> hi, I was facing similar issues some time back, though I was using a
> ruleflow and had >20000 fact objects evaluated daily. The problem I was
> facing was that the running time would get significantly slower day by day.
> The solution that fixed it was to use a statefulSession and call dispose()
> on the session once it is done. This made a single-run slower than before,
> but atleast fixed the problem of exponentially increasing the run-time after
> several days.
>
> This might not necessarily be the ideal solution, so before you move to a
> StatefulSession and call session.dispose() after processing, you might want
> to try to increase the memory for the jvm, if is lower than the size of your
> facts, this might fix it.
>
> It'd be useful to know if somebody has a better solution to Kris's problem.
>
> Thanks,
> - am
>
>
> On Wed, Nov 26, 2008 at 9:35 AM, Kris Nuttycombe <
> kris.nuttycombe at gmail.com> wrote:
>
>> Hi, all,
>>
>> I'm encountering significant problems with running out of memory when
>> executing a small ruleset (only 16 rules, < 100 facts). I've run jmap on the
>> process (which has previously spawned multiple stateless sessions in
>> parallel) and it looks like Drools, and specifically the Eclipse jdt
>> compiler, is seriously eating up memory. The code calling the is fairly
>> simple; one of the initial facts is an accumulator that I refer to later:
>>
>>         StatelessSession session = ruleBase.newStatelessSession();
>>         session.execute(initialFact, accumulator);
>>
>>         accumulator.doSomething();
>>
>> where ruleBase is a shared instance, and initialFact is a base fact that
>> is expanded during rules processing. Is there something I should be doing to
>> dispose the session after use to release the objects created by the drools
>> compiler after use? I'm using a shared RuleBase, and I'm wondering whether
>> it is somehow keeping references around. Are these sorts of object counts to
>> be expected?
>>
>> Thanks,
>>
>> Kris
>> Instance Counts for All Classes (excluding platform) 104827 instances<http://carrot.gaiam.com:7000/instances/0x2aaacf5646f8>of class
>> net.jxta.impl.document.LiteXMLElement$charRange<http://carrot.gaiam.com:7000/class/0x2aaacf5646f8>
>> 26806 instances <http://carrot.gaiam.com:7000/instances/0x2aaacf5631e8>of class
>> net.jxta.impl.document.LiteXMLElement$tagRange<http://carrot.gaiam.com:7000/class/0x2aaacf5631e8>
>> 26733 instances <http://carrot.gaiam.com:7000/instances/0x2aaad66ea7f0>of class
>> org.eclipse.jdt.internal.compiler.lookup.MethodBinding<http://carrot.gaiam.com:7000/class/0x2aaad66ea7f0>
>> 17366 instances <http://carrot.gaiam.com:7000/instances/0x2aaad66e3c88>of class
>> [Lorg.eclipse.jdt.internal.compiler.lookup.TypeBinding;<http://carrot.gaiam.com:7000/class/0x2aaad66e3c88>
>> 12412 instances <http://carrot.gaiam.com:7000/instances/0x2aaacf032560>of class
>> com.sun.enterprise.loader.EJBClassLoader$SentinelInputStream<http://carrot.gaiam.com:7000/class/0x2aaacf032560>
>> 10974 instances <http://carrot.gaiam.com:7000/instances/0x2aaad5504450>of class
>> org.apache.commons.collections.SequencedHashMap$Entry<http://carrot.gaiam.com:7000/class/0x2aaad5504450>
>> 8683 instances <http://carrot.gaiam.com:7000/instances/0x2aaad68c3840> of
>> class org.eclipse.jdt.internal.compiler.codegen.CachedIndexEntry<http://carrot.gaiam.com:7000/class/0x2aaad68c3840>
>> 7708 instances <http://carrot.gaiam.com:7000/instances/0x2aaacf55e9d8> of
>> class net.jxta.impl.document.LiteXMLElement<http://carrot.gaiam.com:7000/class/0x2aaacf55e9d8>
>> 7450 instances <http://carrot.gaiam.com:7000/instances/0x2aaad5c4b6c0> of
>> class org.hibernate.engine.CollectionKey<http://carrot.gaiam.com:7000/class/0x2aaad5c4b6c0>
>> 6836 instances <http://carrot.gaiam.com:7000/instances/0x2aaaced3c3f8> of
>> class org.netbeans.modules.schema2beans.AttrProp<http://carrot.gaiam.com:7000/class/0x2aaaced3c3f8>
>> 6271 instances <http://carrot.gaiam.com:7000/instances/0x2aaad6642d18> of
>> class org.eclipse.jdt.internal.compiler.lookup.FieldBinding<http://carrot.gaiam.com:7000/class/0x2aaad6642d18>
>> 5643 instances <http://carrot.gaiam.com:7000/instances/0x2aaad6796890> of
>> class org.eclipse.jdt.internal.compiler.ast.MessageSend<http://carrot.gaiam.com:7000/class/0x2aaad6796890>
>> 4928 instances <http://carrot.gaiam.com:7000/instances/0x2aaad68a6018> of
>> class org.eclipse.jdt.internal.compiler.util.HashtableOfObject<http://carrot.gaiam.com:7000/class/0x2aaad68a6018>
>> 4895 instances <http://carrot.gaiam.com:7000/instances/0x2aaad66e3fa8> of
>> class [Lorg.eclipse.jdt.internal.compiler.lookup.ReferenceBinding;<http://carrot.gaiam.com:7000/class/0x2aaad66e3fa8>
>> 4546 instances <http://carrot.gaiam.com:7000/instances/0x2aaad669f968> of
>> class org.eclipse.jdt.internal.compiler.lookup.LocalVariableBinding<http://carrot.gaiam.com:7000/class/0x2aaad669f968>
>> 4472 instances <http://carrot.gaiam.com:7000/instances/0x2aaad687a0a8> of
>> class [Lorg.eclipse.jdt.internal.compiler.lookup.LocalVariableBinding;<http://carrot.gaiam.com:7000/class/0x2aaad687a0a8>
>> 4128 instances <http://carrot.gaiam.com:7000/instances/0x2aaad6103ff0> of
>> class org.drools.rule.Pattern<http://carrot.gaiam.com:7000/class/0x2aaad6103ff0>
>> 3899 instances <http://carrot.gaiam.com:7000/instances/0x2aaacf493c90> of
>> class net.jxta.impl.id.UUID.UUID<http://carrot.gaiam.com:7000/class/0x2aaacf493c90>
>> 3867 instances <http://carrot.gaiam.com:7000/instances/0x2aaad5c416c0> of
>> class org.hibernate.engine.EntityEntry<http://carrot.gaiam.com:7000/class/0x2aaad5c416c0>
>> 3731 instances <http://carrot.gaiam.com:7000/instances/0x2aaad5c54d88> of
>> class org.hibernate.collection.PersistentSet<http://carrot.gaiam.com:7000/class/0x2aaad5c54d88>
>> 3685 instances <http://carrot.gaiam.com:7000/instances/0x2aaad66ca420> of
>> class org.eclipse.jdt.internal.compiler.lookup.UnresolvedReferenceBinding<http://carrot.gaiam.com:7000/class/0x2aaad66ca420>
>> 3360 instances <http://carrot.gaiam.com:7000/instances/0x2aaad6c12770> of
>> class org.drools.reteoo.SingleTupleSinkAdapter<http://carrot.gaiam.com:7000/class/0x2aaad6c12770>
>> 3266 instances <http://carrot.gaiam.com:7000/instances/0x2aaacf543dc0> of
>> class net.jxta.document.MimeMediaType<http://carrot.gaiam.com:7000/class/0x2aaacf543dc0>
>> 3264 instances <http://carrot.gaiam.com:7000/instances/0x2aaad6370288> of
>> class org.drools.spi.PatternExtractor<http://carrot.gaiam.com:7000/class/0x2aaad6370288>
>> 3163 instances <http://carrot.gaiam.com:7000/instances/0x2aaad5f8b990> of
>> class org.apache.axis.encoding.TypeMappingImpl$Pair<http://carrot.gaiam.com:7000/class/0x2aaad5f8b990>
>> 3137 instances <http://carrot.gaiam.com:7000/instances/0x2aaad5caad98> of
>> class org.hibernate.collection.PersistentBag<http://carrot.gaiam.com:7000/class/0x2aaad5caad98>
>> 2881 instances <http://carrot.gaiam.com:7000/instances/0x2aaad681b000> of
>> class org.eclipse.jdt.internal.compiler.lookup.MethodScope<http://carrot.gaiam.com:7000/class/0x2aaad681b000>
>> 2688 instances <http://carrot.gaiam.com:7000/instances/0x2aaad6161860> of
>> class org.drools.base.ClassFieldExtractor<http://carrot.gaiam.com:7000/class/0x2aaad6161860>
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>
>
> --
> I'm worst at what I do best, and for this gift I feel blessed
> -- Kurt Cobain
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20081126/1167aa47/attachment.html

More information about the rules-users mailing list