well it is interesting that it means 98% of the time it is GC doing the work. Think about that - that means if it is only 90% GC doing the work (ie 90% of CPU time is spent running the garbage collector - and freeing enough memory to keep it happy) than it would not trip this error - but I am sure no one would be happy with an application that runs that way !<div>
<br><br><div class="gmail_quote">On Sun, Mar 14, 2010 at 12:50 PM, Greg Barton <span dir="ltr"><<a href="mailto:greg_barton@yahoo.com">greg_barton@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Isn't that what Geoffrey just said? :)<br>
<br>
Anyone tried different GC approaches with this? Running with the concurrent collector might be a useful stopgap until the object creation is tamped down.<br>
<br>
--- On Sat, 3/13/10, Michael Neale <<a href="mailto:michael.neale@gmail.com">michael.neale@gmail.com</a>> wrote:<br>
<br>
> From: Michael Neale <<a href="mailto:michael.neale@gmail.com">michael.neale@gmail.com</a>><br>
> Subject: Re: [rules-dev] memory issue (memory leak?) on trunk (probably due true modify changes)<br>
> To: "Rules Dev List" <<a href="mailto:rules-dev@lists.jboss.org">rules-dev@lists.jboss.org</a>><br>
> Cc: "<a href="mailto:rules-dev@lists.jboss.org">rules-dev@lists.jboss.org</a>" <<a href="mailto:rules-dev@lists.jboss.org">rules-dev@lists.jboss.org</a>><br>
> Date: Saturday, March 13, 2010, 5:54 PM<br>
<div><div></div><div class="h5">> Interesting error.<br>
><br>
> One explanation of it: <a href="http://stackoverflow.com/questions/1393486/what-means-the-error-message-java-lang-outofmemoryerror-gc-overhead-limit-excee" target="_blank">http://stackoverflow.com/questions/1393486/what-means-the-error-message-java-lang-outofmemoryerror-gc-overhead-limit-excee</a><br>
><br>
> Sent from my phone.<br>
><br>
> On 13/03/2010, at 8:21 PM, Geoffrey De Smet <<a href="mailto:ge0ffrey.spam@gmail.com">ge0ffrey.spam@gmail.com</a>> <br>
><br>
> wrote:<br>
><br>
> > Hi guys,<br>
> ><br>
> > With great improvements usually come instability for a<br>
> while :)<br>
> > Trunk has memory issue, probably a memory leak of a<br>
> small object.<br>
> ><br>
> > I've run ExmaminationBenchmarkApp with the program arg<br>
> "short" 3 times<br>
> > with the exact same configuration (it is a 100%<br>
> reproducible run):<br>
> ><br>
> > 1) Before the true modify changes with 512m max<br>
> memory<br>
> > Succeeds (100 steps but even 10000 steps work) with no<br>
> slow down.<br>
> ><br>
> > 2) After true modify changes with 512m max memory<br>
> > Fails after 27 steps, starts slowing down at 24<br>
> steps.<br>
> ><br>
> > 3) After true modify changes with 1024m max memory<br>
> > Fails after 57 steps, starts slowing down at 54<br>
> steps.<br>
> ><br>
> > Both 2) and 3) fail with the exception:<br>
> > Exception in thread "main" java.lang.OutOfMemoryError:<br>
> GC overhead <br>
> > limit<br>
> > exceeded<br>
> ><br>
> > So it's not a heap space error or a perm space error,<br>
> but a GC <br>
> > overhead<br>
> > limit instead. Literally this means it's doing more GC<br>
> then running,<br>
> > but 99% of the time this means that the heap is so<br>
> full that the GC <br>
> > has<br>
> > little or no free space to work with.<br>
> > This kind of memory leak happens when the iteration<br>
> make 101 objects <br>
> > but<br>
> > only removes 100 objects and the iteration is run a<br>
> lot (like 10000+ <br>
> > times).<br>
> > You don't get a heap space error, because before that<br>
> happens,<br>
> > there is point where you have a garbaged collected<br>
> free memory for 10<br>
> > objects and a ungarbaged collected free memory for 100<br>
> objects. At <br>
> > that<br>
> > point the GC is running over 98% of the time, which is<br>
> a VM treshold <br>
> > and<br>
> > the error is thrown,<br>
> > even though 10 iterations later you would get a heap<br>
> space error.<br>
> ><br>
> ><br>
> > Logs:<br>
> ><br>
> > 1)<br>
> > TODO<br>
> ><br>
> > 2)<br>
> > 2010-03-13 09:28:36,973 [main] INFO Step index<br>
> (21), time spend <br>
> > (37245)<br>
> > taking step (323 {D180|S9} @ 9 {D210} + 5 {C65} =><br>
> 52 {D210}) out of<br>
> > 1402 accepted moves.<br>
> > 2010-03-13 09:28:38,454 [main] INFO Step index<br>
> (22), time spend <br>
> > (38727)<br>
> > taking step (367 {D190|S40} @ 28 {D210} + 5 {C65}<br>
> => 19 {D210}) out of<br>
> > 1366 accepted moves.<br>
> > 2010-03-13 09:28:39,903 [main] INFO Step index<br>
> (23), time spend <br>
> > (40176)<br>
> > taking step (230 {D120|S15} @ 7 {D210} + 3 {C60} =><br>
> 12 {D210}) out of<br>
> > 1366 accepted moves.<br>
> > 2010-03-13 09:28:43,901 [main] INFO Step index<br>
> (24), time spend <br>
> > (44174)<br>
> > taking step (190 {D90|S6} @ 1 {D210} + 3 {C60}<br>
> <=> 205 {D120|S6} @ 8<br>
> > {D210} + 6 {C111}) out of 1392 accepted moves.<br>
> > 2010-03-13 09:28:50,631 [main] INFO Step index<br>
> (25), time spend <br>
> > (50904)<br>
> > taking step (420 {D180|S6} @ 46 {D210} + 2 {C129}<br>
> => 25 {D210}) out of<br>
> > 1396 accepted moves.<br>
> > 2010-03-13 09:29:06,184 [main] INFO Step index<br>
> (26), time spend <br>
> > (66457)<br>
> > taking step (211 {D180|S53} @ 15 {D210} + 0 {C260}<br>
> => 23 {D210}) out <br>
> > of<br>
> > 1383 accepted moves.<br>
> > 2010-03-13 09:29:46,589 [main] INFO Step index<br>
> (27), time spend<br>
> > (106862) taking step (39 {D120|S15} @ 50 {D210} + 2<br>
> {C129} => 43 <br>
> > {D210})<br>
> > out of 1383 accepted moves.<br>
> > Exception in thread "main" java.lang.OutOfMemoryError:<br>
> GC overhead <br>
> > limit<br>
> > exceeded<br>
> > at<br>
> java.util.HashMap.addEntry(HashMap.java:753)<br>
> > at<br>
> java.util.HashMap.put(HashMap.java:385)<br>
> > at<br>
> java.util.HashMap.putAll(HashMap.java:524)<br>
> > at<br>
> > org.drools.rule.GroupElement$Type.getOuterDeclarations<br>
><br>
> > (GroupElement.java:406)<br>
> > at<br>
> org.drools.rule.GroupElement.getOuterDeclarations<br>
> > (GroupElement.java:109)<br>
> > at<br>
> > org.drools.base.DefaultKnowledgeHelper.getDeclaration<br>
><br>
> > (DefaultKnowledgeHelper.java:213)<br>
> > at<br>
> ><br>
> org.drools.planner.examples.examination.solver.Rule_periodSpread_0DefaultConsequenceInvoker.evaluate(<br>
><br>
> > Rule_periodSpread_0DefaultConsequenceInvoker.java:21)<br>
> > at<br>
> org.drools.common.DefaultAgenda.fireActivation<br>
> > (DefaultAgenda.java:903)<br>
> > at<br>
> org.drools.common.DefaultAgenda.fireNextItem<br>
> > (DefaultAgenda.java:850)<br>
> > at<br>
> org.drools.common.DefaultAgenda.fireAllRules<br>
> > (DefaultAgenda.java:1061)<br>
> > at<br>
> > org.drools.common.AbstractWorkingMemory.fireAllRules<br>
> > (AbstractWorkingMemory.java:741)<br>
> > at<br>
> > org.drools.common.AbstractWorkingMemory.fireAllRules<br>
> > (AbstractWorkingMemory.java:707)<br>
> > ...<br>
> ><br>
> > 3)<br>
> > 2010-03-13 09:47:38,303 [main] INFO Step index<br>
> (55), time spend<br>
> > (114213) taking step (186 {D90|S17} @ 24 {D210} + 5<br>
> {C65} <=> 516<br>
> > {D180|S12} @ 15 {D210} + 0 {C260}) out of 1382<br>
> accepted moves.<br>
> > 2010-03-13 09:48:02,062 [main] INFO Step index<br>
> (56), time spend<br>
> > (137972) taking step (498 {D120|S17} @ 28 {D210} + 0<br>
> {C260} => 1 <br>
> > {C100})<br>
> > out of 1389 accepted moves.<br>
> > 2010-03-13 09:48:35,748 [main] INFO Step index<br>
> (57), time spend<br>
> > (171658) taking step (15 {D135|S96} @ 16 {D210} + 1<br>
> {C100} <=> 164<br>
> > {D180|S48} @ 3 {D210} + 0 {C260}) out of 1378 accepted<br>
> moves.<br>
> > Exception in thread "main" java.lang.OutOfMemoryError:<br>
> GC overhead <br>
> > limit<br>
> > exceeded<br>
> > at<br>
> java.lang.Integer.valueOf(Integer.java:625)<br>
> > at<br>
> ><br>
> org.drools.common.TruthMaintenanceSystem.removeLogicalDependencies<br>
><br>
> > (TruthMaintenanceSystem.java:180)<br>
> > at<br>
> ><br>
> org.drools.common.AbstractWorkingMemory.removeLogicalDependencies<br>
><br>
> > (AbstractWorkingMemory.java:1455)<br>
> > at<br>
> > org.drools.reteoo.RuleTerminalNode.retractLeftTuple<br>
> > (RuleTerminalNode.java:266)<br>
> > at<br>
> ><br>
> org.drools.reteoo.SingleLeftTupleSinkAdapter.doPropagateRetractLeftTuple(<br>
><br>
> > SingleLeftTupleSinkAdapter.java:176)<br>
> > at<br>
> ><br>
> org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateRetractLeftTuple(<br>
><br>
> > SingleLeftTupleSinkAdapter.java:57)<br>
> > at<br>
> > org.drools.reteoo.EvalConditionNode.modifyLeftTuple<br>
> > (EvalConditionNode.java:250)<br>
> > at<br>
> ><br>
> org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateModifyChildLeftTuple(<br>
><br>
> > SingleLeftTupleSinkAdapter.java:198)<br>
> > at<br>
> org.drools.reteoo.JoinNode.modifyLeftTuple(JoinNode.java:304)<br>
> > at<br>
> ><br>
> org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateModifyChildLeftTuple(<br>
><br>
> > SingleLeftTupleSinkAdapter.java:209)<br>
> > at<br>
> org.drools.reteoo.JoinNode.modifyRightTuple(JoinNode.java:221)<br>
> > at<br>
> org.drools.reteoo.BetaNode.modifyObject(BetaNode.java:317)<br>
> > at<br>
> ><br>
> org.drools.reteoo.CompositeObjectSinkAdapter.doPropagateModifyObject<br>
><br>
> > (CompositeObjectSinkAdapter.java:444)<br>
> > at<br>
> ><br>
> org.drools.reteoo.CompositeObjectSinkAdapter.propagateModifyObject<br>
><br>
> > (CompositeObjectSinkAdapter.java:412)<br>
> > at<br>
> org.drools.reteoo.ObjectTypeNode.modifyObject<br>
> > (ObjectTypeNode.java:262)<br>
> > at<br>
> org.drools.reteoo.EntryPointNode.modifyObject<br>
> > (EntryPointNode.java:174)<br>
> > at<br>
> > org.drools.common.AbstractWorkingMemory.update<br>
> > (AbstractWorkingMemory.java:1392)<br>
> > at<br>
> > org.drools.common.AbstractWorkingMemory.update<br>
> > (AbstractWorkingMemory.java:1288)<br>
> ><br>
> ><br>
> > --<br>
> > With kind regards,<br>
> > Geoffrey De Smet<br>
> ><br>
> > _______________________________________________<br>
> > rules-dev mailing list<br>
> > <a href="mailto:rules-dev@lists.jboss.org">rules-dev@lists.jboss.org</a><br>
> > <a href="https://lists.jboss.org/mailman/listinfo/rules-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-dev</a><br>
> _______________________________________________<br>
> rules-dev mailing list<br>
> <a href="mailto:rules-dev@lists.jboss.org">rules-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/rules-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-dev</a><br>
><br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
rules-dev mailing list<br>
<a href="mailto:rules-dev@lists.jboss.org">rules-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Michael D Neale<br>home: <a href="http://www.michaelneale.net">www.michaelneale.net</a><br>blog: <a href="http://michaelneale.blogspot.com">michaelneale.blogspot.com</a><br>
</div>