[rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects
Julian Klein
julianklein at gmail.com
Tue Feb 19 22:32:12 EST 2013
Thanks, Mark. I am avoiding inference by storing the results of rules
execution in memory and then firing the rules against those at a later
point in the program execution. This workaround enables me to more
directly control data flow and use a Stateless Session.
In regards to my original post, I've had a breakthrough in regards to
memory overhead. I switched to a StatelessKnowledgeSession and avoided the
above locking condition. Please ignore the line I mentioned previously as
that one is incorrect. I was getting a lock within the fireAllRules
method. Anyway, I now cache the KnowledgeBuilder and create a new
KnowledgeBase and Session for each thread needing to execute rules. This
stopped the lock condition from occurring when firing rules as there is no
shared rule base across threads. The count of JoinNodeLeftTuple objects
and overall memory usage has dropped dramatically. It seems the root cause
was the caching of my KnowledgeBase object for reuse and the use of a
StatefulKnowledgeSession. Given the complexity of my scenario, I have been
unable to create a simple use case. I suppose at this stage this begs the
question of whether or not it is appropriate to cache the KnowledgeBase
instead of simply the KnowledgeBuilder? I have also set the keep reference
flag set to false for the session config, but something was keeping a
handle on expired sessions anyway. Either that or the stateless session's
footprint is several orders of magnitude smaller.
Thanks.
On Tue, Feb 19, 2013 at 7:08 PM, Mark Proctor <mproctor at codehaus.org> wrote:
>
> On 19 Feb 2013, at 15:57, Julian Klein <julianklein at gmail.com> wrote:
>
> Ok, so I reviewed my code again and found a spot where I am queuing up
> tasks to process data that hadn't been loaded. My application shouldn't
> spawn a rule sessions in the case. I have also moved to a sever with less
> RAM and will begin profiling as soon as possible. Here is where I stand
> as of now:
>
> 1) I am still seeing about 250 million JoinNodeLeftTuples (eats up about
> 30GB of RAM). This greatly outnumbers anything else in the system by a
> factor of 100x.
> 2) Creating a test case will be very difficult as I cannot pinpoint the
> root cause of this issue. I am not sure if it is related to too many rules
> engine, the size of my ruleset, or simply the scale at which I am
> attempting to use Drools. If I pinpoint an actual memory leak, I will
> create a test case and submit.
> 3) I cannot use a StatelessKnowledgeSession as it locks my process up. It
> seems the code in StatelessKnowledgeSessionImpl's constructor creates a
> lock that never releases. No idea on this either for now.
>
> this.ruleBase.lock();
> try {
> if ( ruleBase.getConfiguration().isSequential() ) {
> this.ruleBase.getReteooBuilder().order();
> }
> } finally {
> * this.ruleBase.unlock(); <--- This line is never
> reached for some reason*
> }
>
> I will continue trying to find a root cause or path forward and report
> back. If anything here gives you all a sense of some wrongdoing on my
> behalf, please let me know. I don't fully understand sequential mode as
> the documentation is not approachable from my uninformed perspective. So
> perhaps there is something I am doing in my rules that leads to this lock
> not getting release?
>
> Sequential mode means no inference. If you want to modify or insert facts
> and have them evaluated in the rules, then you don't want sequential mode.
>
>
> Thanks again,
> Julian
>
>
> On Mon, Feb 18, 2013 at 9:13 AM, Julian Klein <julianklein at gmail.com>wrote:
>
>> Unfortunately the server I was using locked up last night and I cannot
>> get access. I'll have to find an interim system where I can run tests and
>> profile the heap. In the meantime, I switched to a Sequential Stateless
>> Session. I don't know the internals of Drools, but since this seems to
>> turn off the left input propagation, does that mean the JoinLeftNode is not
>> used?
>>
>> I am running 5.4 .0 on Linux. If I see the same issues with a stateless
>> approach, I will try the 5.5 approach.
>>
>> Thanks again,
>> Julian
>>
>>
>> On Feb 17, 2013, at 6:19 PM, Mark Proctor <mproctor at codehaus.org> wrote:
>>
>> You'll need to help us a bit here, by creating a test of some sort. "I
>> have 100 rules with 1B JoinLeftNode" doesn't give us much to go on.
>>
>> The problem mentioned earlier was related to windows, which I don't think
>> you are using - or you'd have said so.
>>
>> Also can you tell us what version you are using? Could you maybe try the
>> 5..5.x branch which has a lot of fixes in it (you'll need to build it with
>> maven):
>> https://github.com/droolsjbpm/drools/tree/5.5.x
>>
>> mark
>> On 17 Feb 2013, at 18:27, Julian Klein <julianklein at gmail.com> wrote:
>>
>> Thanks again. I have tried Sequential mode, but somehow my Java based
>> class that I use for function calls no longer receives them. Perhaps there
>> is an error occurring before the call to my class in the "then" section of
>> the rule that is not presenting itself in my logs. Either that or I don't
>> understand sequential mode's limitations well enough. I'll try it one more
>> time and report back on that as well.
>>
>>
>> On Sun, Feb 17, 2013 at 1:17 PM, Wolfgang Laun <wolfgang.laun at gmail.com>wrote:
>>
>>> On 17/02/2013, Julian Klein <julianklein at gmail.com> wrote:
>>> > Ok, I witnessed this with JProfiler. I will try with jmap to make
>>> sure and
>>> > report back later today or tomorrow. Is it safe then to assume that
>>> > discarding (dispose) the session between laps is better from a
>>> throughput
>>> > perspective than re-using it?
>>>
>>> If you need to start each session with a clean slate: yes, dispose is
>>> preferable.
>>>
>>> > Also, would using a Stateless Session
>>> > "perform" better from either a memory or CPU perspective?
>>>
>>> Only if you can run it in "sequential mode" - see Expert, 3.3.7.1.
>>> Sequential Mode
>>>
>>> -W
>>>
>>> >
>>> > Thanks again!
>>> >
>>> _______________________________________________
>>> rules-users mailing list
>>> rules-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20130219/1b56b3b3/attachment.html
More information about the rules-users
mailing list