[rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects

[Mark Proctor]

Typical issues are:
1) Not retracting facts, or setting expiration.
2) Not disposing sessions.
3) Poor discrimination networks that produce too many cross products - put the strongest descriptors first.
4) Making sure you use constraints that exploit indexing (this makes a huge difference for performance)
5) Excessive recursion, cause massive memory consumption.
6) Design your rules to facilitate node sharing better.
7) Drools is single threaded, Each external thread goes through a jdk lock to ensure serial access.

You need to isolate your program down to a few rules, show that it either works or has a problem. And then just build that out from there, until you can demonstrate the problem.

You can't just build a large rule base, and then wonder why there are problems at the end. This is the strongest piece of advice I give to starting a rule based project - as I see this problem happen often. You should build out small and slowly, testing at each stage - for correctness and performance. That way when problems occur, you are able to isolate the problem to a small range of code commits. Never be in a hurry to rush to the finishing line, think tortoise and the hare.

We have other users scaling to millions of facts and hundreds of rules, running in reasonable amount of time.  We do get bugs, and we always work hard to get those isolated, but we need something to allow us to help diagnose the problem.

Mark

On 23 Feb 2013, at 21:10, Julian Klein <julianklein at gmail.com> wrote:

> Although I did see a drop in initial memory usage, things went south again as the run continued past the 2-4 hour mark.  I have poured over my source several times and am starting to loose faith that Drools can provide anything other than a single threaded rules engine.  Although the engine itself is fast, it seems it cannot scale from a memory perspective.  I understand I am not providing much detail or a use case, but perhaps there are some official best practices documented somewhere on how to scale Drools?  I cannot get valuable heap dumps because the heaps are enormous.  I have continuously been told that Drools doesn't perform by colleagues and was hoping they were wrong.  Any pointers or help is greatly appreciated.
> 
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users