[rules-users] Memory issues

Thu Jul 14 05:03:55 EDT 2011

Hi,

it's still guesswork in parts - circumstantial evidence at best.

The purpose of a KnowledgeAgent is to monitor resources for building a
KnowledgeBase, typically by periodic inspection of certain parts of the file
system and reacting to updates. But you set all scan.X properties to false,
turning the feature completely off.

This means that you use a KnowledgeAgent merely as a convenience tool for
creating a rule base. This is a sign of laziness (one of the three great
virtues of a programmer <http://c2.com/cgi/wiki?LazinessImpatienceHubris>)
but even it must be exercised with care.  However: why does the change Set
XML contain just the compiled binary? This means that stuff is compiled
elsewhere, and only the result copied to a place that's being monitored by -
which part of your application?

Agents are notoriously hard to kill, and the KA is one of these. If, for
instance you start resource change monitoring for an agent, it'll run a
thread that keeps a reference to the KA. So, just wiping a Map will not rid
the program of the KA objects, and this is bound to keep a lot of other
biggies alive.

Did you ever monitor to see running threads? Or existing objects per class?

HTH
-W

2011/7/13 Jeffrey Schneller <jeffrey.schneller at envisa.com>

> Yes.  The 18 KnowledgeAgents produce 18 StatefulKnowledgeSessions when they
> are needed.  The session has dispose() called on it when it is not needed
> any longer.****
>
> ** **
>
> The Map<X,Y> maps the following:****
>
> ** **
>
> X = product name****
>
> Y = the Knowledge Agent associated to a product.****
>
> ** **
>
> Code does the following when rules need to be run against a product.****
>
> ** **
>
> **1)      **Get KnowledgeAgent out of Map based on product name****
>
> **2)      **Create a StatefulKnowledgeSession****
>
> **3)      **Insert the facts about the product****
>
> **4)      **Fire all rules****
>
> **5)      **Dispose of StatefulKnowledgeSession****
>
> ** **
>
> We have no problems with the 5 steps that are followed.  The rules are
> configuration and pricing rules for each product.  The facts are the options
> specified for a particular product. The rules compute the price of the
> product as specified by the desired options and any incompatible options
> selected [very similar to what you would see on car manufacturers website
> for pricing a particular car]  If there is a better way to do this, I would
> be interested in knowing.****
>
> ** **
>
> Jeff****
>
> ** **
>
> ** **
>
> *From:* rules-users-bounces at lists.jboss.org [mailto:
> rules-users-bounces at lists.jboss.org] *On Behalf Of *Wolfgang Laun
> *Sent:* Wednesday, July 13, 2011 1:27 PM
> *To:* Rules Users List
> *Subject:* Re: [rules-users] Memory issues****
>
> ** **
>
> This is strange - do you mean that those 18 KnowledgeAgents stored in that
> Map produce 18 different StatefulKnowledgeSessions?
>
> What does the Map<X,Y> map? X? Y?
>
> -W****
>
> 2011/7/13 Jeffrey Schneller <jeffrey.schneller at envisa.com>****
>
> I am currently running version 5.1.0M2.  I have 18 different binary
> rulesets [packages] that are stored on the local filesystem.  The rulesets
> range in size from 4K to 131Mb. ****
>
>  ****
>
> When my application starts I load each ruleset into a knowledgeAgent and
> store the knowledgeAgent into a hashmap which is stored in memory.   They
> are loaded using the following xml:****
>
>  ****
>
> <?xml version=\"1.0\" encoding=\"UTF-8\"?>****
>
> <change-set xmlns='http://drools.org/drools-5.0/change-set' xmlns:xs='
> http://www.w3.org/2001/XMLSchema-instance' xs:schemaLocation='
> http://drools.org/drools-5.0/change-set.xsd'>****
>
> <add>****
>
> <resource source='file://PATH_TO_BINARY' type='PKG' />****
>
> </add>****
>
> </change-set>****
>
>  ****
>
> The knowledgeAgentConfiguration is setup as:****
>
> drools.agent.scanDirectories = false****
>
> drools.agent.newInstance = false****
>
> drools.agent.scanResources = false****
>
> drools.agent.monitorChangeEvents = false****
>
>  ****
>
> After the all the agents are loaded and stored in the hashmap the total
> memory used in the app is 6.2GB.  ****
>
>  ****
>
> Once the application is running, if a business user needs to update a set
> of rules there is a process in place to refresh all the knowledgeAgents with
> new rulesets.  In a new thread. the code clears the hashmap, sets the
> hashmap to null, calls for garbage collection to occur, and it then
> constructs a new hashmap and loads all the rules as it does at application
> startup. ****
>
>  ****
>
> The knowledge agent is logging the following when the ruleset is loaded:**
> **
>
> KnowledgeAgent applying ChangeSet****
>
> KnowledgeAgent performing an incremental build of the ChangeSet****
>
> KnowledgeAgent incremental build of KnowledgeBase finished and in use****
>
>  ****
>
>  ****
>
> It takes a large amount of time to do this process [double to triple the
> time of application startup]. After this occurs the memory used by the app
> increases [to nearly double the original size 12.2GB] and the CPU is maxed
> out.  After some time the CPU usage drops and the app is usable again but
> the memory never decreases.  If a business user refreshes the rules again,
> the memory increases again and the CPU is maxed out and the app becomes
> unresponsive because the memory is completely maxed out.****
>
>  ****
>
> There is obviously a memory leak somewhere and a large one at that.  Is
> this the proper way to be caching KnowledgeAgents into memory so the
> rulessets don’t need to be re-loaded everytime a knowledgeAgent is run?  Is
> my problem the use of the hashmap?  Do I need to remove the original
> knowledgeAgents that were in the hashmap?****
>
>  ****
>
>  ****
>
> Thanks for any help.****
>
>  ****
>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users****
>
> ** **
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20110714/5d9d3f3d/attachment.html 