[rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects

[Julian Klein]

Ok, so I reviewed my code again and found a spot where I am queuing up
tasks to process data that hadn't been loaded.  My application shouldn't
spawn a rule sessions in the case.  I have also moved to a sever with less
RAM and will begin profiling as soon as possible.   Here is where I stand
as of now:

1) I am still seeing about 250 million JoinNodeLeftTuples (eats up about
30GB of RAM).  This greatly outnumbers anything else in the system by a
factor of 100x.
2) Creating a test case will be very difficult as I cannot pinpoint the
root cause of this issue.  I am not sure if it is related to too many rules
engine, the size of my ruleset, or simply the scale at which I am
attempting to use Drools.  If I pinpoint an actual memory leak, I will
create a test case and submit.
3) I cannot use a StatelessKnowledgeSession as it locks my process up.  It
seems the code in StatelessKnowledgeSessionImpl's constructor creates a
lock that never releases.  No idea on this either for now.

            this.ruleBase.lock();
            try {
                if ( ruleBase.getConfiguration().isSequential() ) {
                    this.ruleBase.getReteooBuilder().order();
                }
            } finally {
*                this.ruleBase.unlock();   <--- This line is never reached
for some reason*
            }

I will continue trying to find a root cause or path forward and report
back.  If anything here gives you all a sense of some wrongdoing on my
behalf, please let me know.  I don't fully understand sequential mode as
the documentation is not approachable from my uninformed perspective.  So
perhaps there is something I am doing in my rules that leads to this lock
not getting release?

Thanks again,
Julian


On Mon, Feb 18, 2013 at 9:13 AM, Julian Klein <julianklein at gmail.com> wrote:

> Unfortunately the server I was using locked up last night and I cannot get
> access.  I'll have to find an interim system where I can run tests and
> profile the heap.  In the meantime, I switched to a Sequential Stateless
> Session.  I don't know the internals of Drools, but since this seems to
> turn off the left input propagation, does that mean the JoinLeftNode is not
> used?
>
> I am running 5.4 .0 on Linux.  If I see the same issues with a stateless
> approach, I will try the 5.5 approach.
>
> Thanks again,
> Julian
>
>
> On Feb 17, 2013, at 6:19 PM, Mark Proctor <mproctor at codehaus.org> wrote:
>
> You'll need to help us a bit here, by creating a test of some sort. "I
> have 100 rules with 1B JoinLeftNode" doesn't give us much to go on.
>
> The problem mentioned earlier was related to windows, which I don't think
> you are using - or you'd have said so.
>
> Also can you tell us what version you are using? Could you maybe try the
> 5..5.x branch which has a lot of fixes in it (you'll need to build it with
> maven):
> https://github.com/droolsjbpm/drools/tree/5.5.x
>
> mark
> On 17 Feb 2013, at 18:27, Julian Klein <julianklein at gmail.com> wrote:
>
> Thanks again. I have tried Sequential mode, but somehow my Java based
> class that I use for function calls no longer receives them.  Perhaps there
> is an error occurring before the call to my class in the "then" section of
> the rule that is not presenting itself in my logs.  Either that or I don't
> understand sequential mode's limitations well enough.  I'll try it one more
> time and report back on that as well.
>
>
> On Sun, Feb 17, 2013 at 1:17 PM, Wolfgang Laun <wolfgang.laun at gmail.com>wrote:
>
>> On 17/02/2013, Julian Klein <julianklein at gmail.com> wrote:
>> > Ok, I witnessed this with JProfiler.  I will try with jmap to make sure
>> and
>> > report back later today or tomorrow.   Is it safe then to assume that
>> > discarding (dispose) the session between laps is better from a
>> throughput
>> > perspective than re-using it?
>>
>> If you need to start each session with a clean slate: yes, dispose is
>> preferable.
>>
>> >  Also, would using a Stateless Session
>> > "perform" better from either a memory or CPU perspective?
>>
>> Only if you can run it in "sequential mode" - see Expert, 3.3.7.1.
>> Sequential Mode
>>
>> -W
>>
>> >
>> > Thanks again!
>> >
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20130219/27bdb179/attachment.html