[rules-users] Memory issues

Thu Jul 14 11:15:08 EDT 2011

Wolfgang -

Thanks for your response.  I have removed the KnowledgeAgent and changed
to build the KnowledgeBase without the use of an agent.  The
ChangeSetXML contains just the compiled binary because there is another
process which pulls the compiled binary from the Guvnor.  This allows
the business users to update the rules in the Guvnor and test in a
staging application.  When the rules are ready to be moved to
production, a process is kicked off that pulls all the binaries from the
Guvnor.  This allows the business to know exactly when the rules were
moved to production and by who [in case there was a problem].

Thanks again.

From: rules-users-bounces at lists.jboss.org
[mailto:rules-users-bounces at lists.jboss.org] On Behalf Of Wolfgang Laun
Sent: Thursday, July 14, 2011 5:04 AM
To: Rules Users List
Subject: Re: [rules-users] Memory issues

Hi,

it's still guesswork in parts - circumstantial evidence at best.

The purpose of a KnowledgeAgent is to monitor resources for building a
KnowledgeBase, typically by periodic inspection of certain parts of the
file system and reacting to updates. But you set all scan.X properties
to false, turning the feature completely off.

This means that you use a KnowledgeAgent merely as a convenience tool
for creating a rule base. This is a sign of laziness (one of the three
great virtues of a programmer
<http://c2.com/cgi/wiki?LazinessImpatienceHubris> ) but even it must be
exercised with care.  However: why does the change Set XML contain just
the compiled binary? This means that stuff is compiled elsewhere, and
only the result copied to a place that's being monitored by - which part
of your application?

Agents are notoriously hard to kill, and the KA is one of these. If, for
instance you start resource change monitoring for an agent, it'll run a
thread that keeps a reference to the KA. So, just wiping a Map will not
rid the program of the KA objects, and this is bound to keep a lot of
other biggies alive.

Did you ever monitor to see running threads? Or existing objects per
class?

HTH
-W

2011/7/13 Jeffrey Schneller <jeffrey.schneller at envisa.com>

Yes.  The 18 KnowledgeAgents produce 18 StatefulKnowledgeSessions when
they are needed.  The session has dispose() called on it when it is not
needed any longer.

The Map<X,Y> maps the following:

X = product name

Y = the Knowledge Agent associated to a product.

Code does the following when rules need to be run against a product.

1)      Get KnowledgeAgent out of Map based on product name

2)      Create a StatefulKnowledgeSession

3)      Insert the facts about the product

4)      Fire all rules

5)      Dispose of StatefulKnowledgeSession

We have no problems with the 5 steps that are followed.  The rules are
configuration and pricing rules for each product.  The facts are the
options specified for a particular product. The rules compute the price
of the product as specified by the desired options and any incompatible
options selected [very similar to what you would see on car
manufacturers website for pricing a particular car]  If there is a
better way to do this, I would be interested in knowing.

Jeff

From: rules-users-bounces at lists.jboss.org
[mailto:rules-users-bounces at lists.jboss.org] On Behalf Of Wolfgang Laun
Sent: Wednesday, July 13, 2011 1:27 PM
To: Rules Users List
Subject: Re: [rules-users] Memory issues

This is strange - do you mean that those 18 KnowledgeAgents stored in
that Map produce 18 different StatefulKnowledgeSessions?

What does the Map<X,Y> map? X? Y?

-W

2011/7/13 Jeffrey Schneller <jeffrey.schneller at envisa.com>

I am currently running version 5.1.0M2.  I have 18 different binary
rulesets [packages] that are stored on the local filesystem.  The
rulesets range in size from 4K to 131Mb. 

When my application starts I load each ruleset into a knowledgeAgent and
store the knowledgeAgent into a hashmap which is stored in memory.
They are loaded using the following xml:

<?xml version=\"1.0\" encoding=\"UTF-8\"?>

<change-set xmlns='http://drools.org/drools-5.0/change-set'
xmlns:xs='http://www.w3.org/2001/XMLSchema-instance'
xs:schemaLocation='http://drools.org/drools-5.0/change-set.xsd'>

<add>

<resource source='file://PATH_TO_BINARY' type='PKG' />

</add>

</change-set>

The knowledgeAgentConfiguration is setup as:

drools.agent.scanDirectories = false

drools.agent.newInstance = false

drools.agent.scanResources = false

drools.agent.monitorChangeEvents = false

After the all the agents are loaded and stored in the hashmap the total
memory used in the app is 6.2GB.  

Once the application is running, if a business user needs to update a
set of rules there is a process in place to refresh all the
knowledgeAgents with new rulesets.  In a new thread. the code clears the
hashmap, sets the hashmap to null, calls for garbage collection to
occur, and it then constructs a new hashmap and loads all the rules as
it does at application startup. 

The knowledge agent is logging the following when the ruleset is loaded:

KnowledgeAgent applying ChangeSet

KnowledgeAgent performing an incremental build of the ChangeSet

KnowledgeAgent incremental build of KnowledgeBase finished and in use

It takes a large amount of time to do this process [double to triple the
time of application startup]. After this occurs the memory used by the
app increases [to nearly double the original size 12.2GB] and the CPU is
maxed out.  After some time the CPU usage drops and the app is usable
again but the memory never decreases.  If a business user refreshes the
rules again, the memory increases again and the CPU is maxed out and the
app becomes unresponsive because the memory is completely maxed out.

There is obviously a memory leak somewhere and a large one at that.  Is
this the proper way to be caching KnowledgeAgents into memory so the
rulessets don't need to be re-loaded everytime a knowledgeAgent is run?
Is my problem the use of the hashmap?  Do I need to remove the original
knowledgeAgents that were in the hashmap?

Thanks for any help.

_______________________________________________
rules-users mailing list
rules-users at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
rules-users at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20110714/3960b3a5/attachment.html 