[rules-users] Out of Memory Issue of JBoss Enginedeployed onIBMWAS 6.0.2.7

[Edson Tirelli]

   Joel,

   That is a really difficult question to answer. :) It depends on the 
profile of your rule and fact bases I guess.
   Drools 3 is a complete Rete implementation with lots of improvements. 
Drools 2.x was a partial implementation. Some of the improvements in 3.x 
do have a memory x performance trade-off, but others simply avoid 
additional work done in 2.1 that caused more objects to be allocated and 
more memory to be allocated.

   It is easier to compare JBoss Rules 3.0.x with JBoss Rules 3.1 
(trunk). They are equivalent engines, but we put a lot of effort on 
resource optimization (both CPU and memory) in 3.1 and it is now more 
optimized and with a smaller memory footprint, even with shadow facts.

   Comparing 2.x with 3.0 must be made case by case I guess. I never tried.

    []s
    Edson


Joel G. Rivera-González wrote:

> edson...
> compared with previus drools versions..
> is drools 3 more memory intensive than version 2.1?
>
> at least on 2.5 my rules take over 250 megs..
> with 2.1 it starts with around 100...
>
>
>  
> Joel G. Rivera-Gonzalez
> PRT
>
>
> ----- Original Message ----
> From: Edson Tirelli <tirelli at post.com>
> To: Rules Users List <rules-users at lists.jboss.org>
> Sent: Tuesday, January 16, 2007 4:45:57 PM
> Subject: Re: [rules-users] Out of Memory Issue of JBoss Enginedeployed 
> onIBMWAS 6.0.2.7
>
>   Dejia,
>
> Just to explain what we are talking about: antlr is the parser generator
> used by JBoss Rules. So, it generates the code to parse the DRL files.
> One part of the parser is a DFA that tries to predict which lexer rule
> should be used by the lexer. What you see in the stack trace, is not the
> same class loaded hundreds of times, but it is each of the DFA and lexer
> inner classes.
> There is not an easy way to "control" what antlr does when generating
> the DFA and it can indeed use huge amounts of memory as it now has a
> variable lookahed (LL(*)). We re-wrote some lexer rules for the new
> version to workaround and reduce DFA size and memory requirements, but
> it is not a simple thing to do.
>
> Ok, this is just a highlevel overview.
> In your specific case, my recomendation is that we work together toward
> a possible solution that may or may not require the release of a new
> version. This is because simply upgrading the antlr version may not
> solve the issue. It may require some grammar rule rewriting or it may
> require reconfiguration of your deploy environment.
>
> So, to start with, can you provide:
>
> * your full stacktrace for the out of memory error?
>
> * your hardware and software environment, specially your memory settings
> for the jvm (in websphere it is called server if I remember correctly)
> where you are deploying the application?
>
> * information about the rulebase you are using? (number of rules, size
> of the rule files, samples of rules that may be offending memory the most)
>
> I'm asking this because we have other cases of rulebases successfully
> deployed to WebSphere environment without memory problems, so we need to
> check if rulebase size is the only difference between your environment
> and other environments or if there is also issues regarding type of
> rules you are writing that are particularly stressing the lexer.
>
> []s
> Edson
>
>
> Dejia Wang wrote:
>
> > Thanks Mark. However, with the OOM issue in JBoss Rules 3.0.5, we
> > cannot really use it in enterprise environment. It would be good if we
> > can get 3.0.6 out sooner.
> >
> > BTW, have your guys check in the fix? Is there anything we can help to
> > get 3.0.6 out sooner?
> >
> > Thanks- Dejia
> >
> > ------------------------------------------------------------------------
> >
> > *From:* Mark Proctor [mailto:mproctor at codehaus.org]
> > *Sent:* Monday, January 15, 2007 6:33 PM
> > *To:* Rules Users List
> > *Subject:* Re: [rules-users] Out of Memory Issue of JBoss
> > Enginedeployed onIBMWAS 6.0.2.7
> >
> > Antlr 2.7.7 is just for the StringTemplate stuff, we are actually
> > using Antlr 3. You could attempt to regenerate the grammar again with
> > latest versions of antlrworks and antlr, although I'm not sure if it
> > works without updates to the drl. Not sure when 3.0.6 will be out,
> > probably not for a few weeks.
> >
> > Mark
> > Dejia Wang wrote:
> >
> > BTW, I noticed that current stable version of antlr is 2.7.7 (released
> > November 1, 2006). Rule 3.0.5 has 2.7.6.
> >
> > Edson, will antlr 2.7.7 work?
> >
> > Thanks- Dejia
> >
> > ------------------------------------------------------------------------
> >
> > *From:* Dejia Wang
> > *Sent:* Monday, January 15, 2007 5:40 PM
> > *To:* 'Rules Users List'
> > *Subject:* RE: [rules-users] Out of Memory Issue of JBoss Engine
> > deployed onIBMWAS 6.0.2.7
> >
> > This is good news. Thanks very much, Mark and Edson! BTW, when will
> > 3.0.6 release be out?
> >
> > - Dejia
> >
> > ------------------------------------------------------------------------
> >
> > *From:* Mark Proctor [mailto:mproctor at codehaus.org]
> > *Sent:* Monday, January 15, 2007 4:33 PM
> > *To:* Rules Users List
> > *Subject:* Re: [rules-users] Out of Memory Issue of JBoss Engine
> > deployed onIBMWAS 6.0.2.7
> >
> > I believe this was a bug in antlr that is now fixed. I'll see if we
> > can get a 3.0.6 release done some time updated to the latest antlr.
> > Edson knows more about this, as it was a bug he investigated.
> >
> > Mark
> > Dejia Wang wrote:
> >
> > BTW, I am using the JBoss Rules 3.0.5.
> >
> > ------------------------------------------------------------------------
> >
> > *From:* Dejia Wang
> > *Sent:* Monday, January 15, 2007 3:56 PM
> > *To:* rules-users at lists.jboss.org <mailto:rules-users at lists.jboss.org>
> > *Subject:* [rules-users] Out of Memory Issue of JBoss Engine deployed
> > on IBMWAS 6.0.2.7
> > *Importance:* High
> >
> > Hi All,
> >
> > I ran into OOM (Out of Memory) issue of JBoss Rule Engine deployed on
> > IBM WAS 6.0.2.7. Below is the log
> >
> > …
> >
> > [Loaded org.antlr.runtime.DFA from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/xxx.ear/lib/antlr-3.0ea8.jar 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5Cxxx.ear%5Clib%5Cantlr-3.0ea8.jar>]
> >
> > [Loading superclass and interfaces of org/antlr/runtime/DFA]
> >
> > [Preparing org/antlr/runtime/DFA]
> >
> > [Loaded org.antlr.runtime.DFA$State from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/xxxApplications.ear/lib/antlr-3.0ea8.jar 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5CxxxApplications.ear%5Clib%5Cantlr-3.0ea8.jar>]
> >
> > [Loading superclass and interfaces of org/antlr/runtime/DFA$State]
> >
> > [Loaded org.drools.lang.RuleParserLexer$1 from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/xxxApplications.ear/lib/drools-compiler-3.0.jar 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5CxxxApplications.ear%5Clib%5Cdrools-compiler-3.0.jar>]
> >
> > [Loading superclass and interfaces of org/drools/lang/RuleParserLexer$1]
> >
> > [Loaded org.drools.lang.RuleParserLexer$2 from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/ 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5C> 
>
> > xxxApplications.ear/lib/drools-compiler-3.0.jar]
> >
> > [Loading superclass and interfaces of org/drools/lang/RuleParserLexer$2]
> >
> > [Loaded org.drools.lang.RuleParserLexer$3 from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/ 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5C> 
>
> > xxxApplications.ear/lib/drools-compiler-3.0.jar]
> >
> > [Loading superclass and interfaces of org/drools/lang/RuleParserLexer$3]
> >
> > [Loaded org.drools.lang.RuleParserLexer$4 from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/ 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5C> 
>
> > xxxApplications.ear/lib/drools-compiler-3.0.jar]
> >
> > [Loading superclass and interfaces of org/drools/lang/RuleParserLexer$4]
> >
> > ….
> >
> > [Loaded org.drools.lang.RuleParserLexer$228 from
> > 
> file:/D:/WebSphere/AppServer/profiles/AppSrv01/installedApps/xxxNode02Cell/xxxApplications.ear/lib/drools-compiler-3.0.jar 
>
> > 
> <file:///D:%5CWebSphere%5CAppServer%5Cprofiles%5CAppSrv01%5CinstalledApps%5CxxxNode02Cell%5CxxxApplications.ear%5Clib%5Cdrools-compiler-3.0.jar>]
> >
> > [Loading superclass and interfaces of 
> org/drools/lang/RuleParserLexer$228]
> >
> > …
> >
> > It looks like it loaded the same class hundreds of times until it
> > exhausts the memory.
> >
> > Any idea why this happens and how to fix this?
> >
> > Thanks- Dejia
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >------------------------------------------------------------------------
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >_______________________________________________
> >
> >rules-users mailing list
> >
> >rules-users at lists.jboss.org <mailto:rules-users at lists.jboss.org>
> >
> >https://lists.jboss.org/mailman/listinfo/rules-users
> >
> >  
> >
> >
> >
> >
> >
> >------------------------------------------------------------------------
> >
> >
> >
> >
> >_______________________________________________
> >
> >rules-users mailing list
> >
> >rules-users at lists.jboss.org <mailto:rules-users at lists.jboss.org>
> >
> >https://lists.jboss.org/mailman/listinfo/rules-users
> >
> >  
> >
> >------------------------------------------------------------------------
> >
> >_______________________________________________
> >rules-users mailing list
> >rules-users at lists.jboss.org
> >https://lists.jboss.org/mailman/listinfo/rules-users
> >  
> >
>
>
> -- 
> Edson Tirelli
> Software Engineer - JBoss Rules Core Developer
> Office: +55 11 3124-6000
> Mobile: +55 11 9218-4151
> JBoss, a division of Red Hat @ www.jboss.com <http://www.jboss.com>
>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>------------------------------------------------------------------------
>
>_______________________________________________
>rules-users mailing list
>rules-users at lists.jboss.org
>https://lists.jboss.org/mailman/listinfo/rules-users
>  
>


-- 
 Edson Tirelli
 Software Engineer - JBoss Rules Core Developer
 Office: +55 11 3124-6000
 Mobile: +55 11 9218-4151
 JBoss, a division of Red Hat @ www.jboss.com