[rules-dev] Possible memory leak in Drools 6.0.1
Mark Proctor
mproctor at codehaus.org
Wed Apr 2 07:39:23 EDT 2014
not sure why this would have a memory leak, it all looks quite straight forward joins.
I assume you are using stateless sessions and inserting just a single Position instance? Is there a different number of RT’s after a single Position insert?
A small tip, you don’t need so many patterns. All those constraints can go in a single Pattern.
Mark
On 2 Apr 2014, at 11:20, kellyajp <kellyajp at yahoo.co.uk> wrote:
> Morning Mark,
>
> That was a useful exercise!!, from a sampled set it looks like the RightTuple instances are coming from a single rule package (nccf).
>
> This rule package comes from a large decision table of that is maintained by our business users. This results in a large DRL file being generated containing 2371 rules.
>
> Below I have included some of the rules that were referenced in the data structures, I have also attached the generated DRL file.
>
> The exact same file processes in Drools 5 but fails in Drools 6.
>
> 31usd_lng_corp_bnd_cr_bbb_usd_over0y_under5y_2
> // rule values at A650, header at A644
> rule "31usd_lng_corp_bnd_cr_bbb_usd_over0y_under5y_2"
> salience 64886
> when
> p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
> Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate)
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+157680000000)
> Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
> Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
> Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
> Position(positionDetail.overallCreditQualityStep == 2)
> Position(positionDetail.valuationType in ('Valuation','Both'))
> Position(positionDetail.riskProductType == 'UNKNOWN')
> then
> result.add(classification=new ActionParametersImpl('BaselineClassification'));
> classification.setBaseline('NCCF');
> classification.setLineId("NCCF0100579");
> classification.setObject(p);
> classification.setPositionId(p.getPositionId());
> classification.setDataStatus(p.getDataStatus());
> classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
> classification.setRulePath(ruleHistory.getActivationPath());
> classification.add(a=new ActionParametersImpl( 'BaselineValue'));
> a.setValueName( 'Balance');
> a.setValue( p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
> a.setBaselineType( 'MB');
> a.setValue(p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> end
>
>
> 31usd_lng_corp_bnd_cr_bbb_usd_over0y_under10y_2
> // rule values at A652, header at A644
> rule "31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_2"
> salience 64884
> when
> p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
> Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+157680000000)
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
> Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
> Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
> Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
> Position(positionDetail.overallCreditQualityStep == 2)
> Position(positionDetail.valuationType in ('Valuation','Both'))
> Position(positionDetail.riskProductType == 'UNKNOWN')
> then
> result.add(classification=new ActionParametersImpl('BaselineClassification'));
> classification.setBaseline('NCCF');
> classification.setLineId("NCCF0100581");
> classification.setObject(p);
> classification.setPositionId(p.getPositionId());
> classification.setDataStatus(p.getDataStatus());
> classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
> classification.setRulePath(ruleHistory.getActivationPath());
> classification.add(a=new ActionParametersImpl( 'BaselineValue'));
> a.setValueName( 'Balance');
> a.setValue( p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
> a.setBaselineType( 'MB');
> a.setValue(p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> end
>
>
> 31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_1
> // rule values at A651, header at A644
> rule "31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_1"
> salience 64885
> when
> p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
> Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+157680000000)
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
> Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
> Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
> Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
> Position(positionDetail.overallCreditQualityStep == 2)
> Position(positionDetail.valuationType in ('Valuation','Both'))
> Position(positionDetail.riskProductType != 'UNKNOWN')
> then
> result.add(classification=new ActionParametersImpl('BaselineClassification'));
> classification.setBaseline('NCCF');
> classification.setLineId("NCCF0100581");
> classification.setObject(p);
> classification.setPositionId(p.getPositionId());
> classification.setDataStatus(p.getDataStatus());
> classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
> classification.setRulePath(ruleHistory.getActivationPath());
> classification.add(a=new ActionParametersImpl( 'BaselineValue'));
> a.setValueName( 'Balance');
> a.setValue( p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
> a.setCashflowType( 'SETTLEMENT, Settlement');
> a.setList(p.getPositionDetail().getCashFlows());
> a.setDefaultDaysToCashFlow (1);
> classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
> a.setBaselineType( 'CB');
> a.setCashflowType( 'Principal, Interest');
> a.setList(p.getPositionDetail().getCashFlows());
> a.setDefaultDaysToCashFlow (15000);
> classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
> a.setBaselineType( 'MB');
> a.setValue(p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
> a.setDate(p.getPositionDetail().getInstrument().getMaturityDate());
> a.setDefaultDaysToCashFlow (15000);
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> end
>
>
> 08eur_la_lng_qual_mkt_sec_agncy_dbt_7y_Under_10Y_5
> // rule values at A954, header at A921
> rule "08eur_la_lng_qual_mtk_sec_agncy_dbt_7Y_under_10Y_5"
> salience 64582
> when
> p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
> Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
> Position(positionDetail.valuationType in ('Valuation','Both'))
> Position(positionDetail.instrument != null && positionDetail.instrument.countryOfDomicile in ('AT','BE','BG','CY','CZ','DE','DK','EE','ES','FI','FR' ,'GB','GR' ,'HU' ,'IE' ,'IS' ,'IT' ,'LI' ,'LT' ,'LU' ,'LV' ,'MT' ,'NL' ,'NO' ,'PL' ,'PT' ,'RO' ,'SE' ,'SI' ,'SK'))
> Position(positionDetail.instrument != null && positionDetail.instrument.countryOfIncorporation in ('AT' ,'BE' ,'BG' , 'CY' ,'CZ' ,'DE' , 'DK' ,'EE' ,'ES', 'FI' ,'FR' ,'GB' , 'GR' ,'HU' ,'IE' , 'IS' ,'IT' ,'LI' , 'LT' ,'LU' ,'LV' , 'MT' ,'NL' ,'NO' , 'PL' ,'PT' ,'RO','SE' ,'SI' ,'SK'))
> Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "EUR")
> Position(positionDetail.instrument != null && positionDetail.instrument.collateralType in ('govt liquid gtd','us govt guarant','u.s. government guarantee'))
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+220752000000)
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
> Position(positionDetail.valuationType in ('Valuation','Both'))
> Position(positionDetail.riskProductType != 'UNKNOWN')
> then
> result.add(classification=new ActionParametersImpl('BaselineClassification'));
> classification.setBaseline('NCCF');
> classification.setLineId("NCCF0100405");
> classification.setObject(p);
> classification.setPositionId(p.getPositionId());
> classification.setDataStatus(p.getDataStatus());
> classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
> classification.setRulePath(ruleHistory.getActivationPath());
> classification.add(a=new ActionParametersImpl( 'BaselineValue'));
> a.setValueName( 'Balance');
> a.setValue( p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
> a.setCashflowType( 'SETTLEMENT, Settlement');
> a.setList(p.getPositionDetail().getCashFlows());
> a.setDefaultDaysToCashFlow (1);
> classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
> a.setBaselineType( 'CB');
> a.setCashflowType( 'Principal, Interest');
> a.setList(p.getPositionDetail().getCashFlows());
> a.setDefaultDaysToCashFlow (15000);
> classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
> a.setBaselineType( 'MB');
> a.setValue(p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
> a.setDate(p.getPositionDetail().getInstrument().getMaturityDate());
> a.setDefaultDaysToCashFlow (15000);
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> end
>
> 18cad_lng_sec_iss_prov_over10Y_under35Y_2
> // rule values at A152, header at A138
> rule "18cad_lng_sec_iss_prov_over10y_under35y_2"
> salience 65384
> when
> p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
> Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+315360000000)
> Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+1103760000000)
> Position(positionDetail.instrument != null && positionDetail.instrument.issuerTicker in ('BRCOL','BcpIPB','ALTA','SCDA'
> ,'MP','MANPAC','MPTB','ONTW'
> ,'ONT','ONTB','ONTPAC','Q'
> ,'QTB','QUEPAC','QW','NBRNS'
> ,'NS','PRINCE','NF','NFTB'))
> Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "CAD")
> Position(positionDetail.instrument != null && positionDetail.instrument.issuerIndustry == "govt regional")
> Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "govt")
> Position(positionDetail.valuationType in ('Valuation','Both'))
> Position(positionDetail.riskProductType == 'UNKNOWN')
> then
> result.add(classification=new ActionParametersImpl('BaselineClassification'));
> classification.setBaseline('NCCF');
> classification.setLineId("NCCF0100341");
> classification.setObject(p);
> classification.setPositionId(p.getPositionId());
> classification.setDataStatus(p.getDataStatus());
> classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
> classification.setRulePath(ruleHistory.getActivationPath());
> classification.add(a=new ActionParametersImpl( 'BaselineValue'));
> a.setValueName( 'Balance');
> a.setValue( p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
> a.setBaselineType( 'MB');
> a.setValue(p.getPositionDetail().getValuation().getMtmValue());
> a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
> a.setNettingType('Security');
> a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
> a.setBucket('Total');
> end
>
>
>
>
>
> From: Mark Proctor [via Drools] <[hidden email]>
> To: kellyajp <[hidden email]>
> Sent: Tuesday, 1 April 2014, 14:10
> Subject: Re: Possible memory leak in Drools 6.0.1
>
> any news?
>
> Mark
> On 30 Mar 2014, at 13:38, Mark Proctor <[hidden email]> wrote:
>
>
>
> On 30 Mar 2014, at 13:26, kellyajp <[hidden email]> wrote:
>
>
> I'll get that to you on Monday
>
> On 30 Mar 2014, at 12:59, "Mark Proctor [via Drools]" <<a href="x-msg://133/user/SendEmail.jtp?type=node&node=4029027&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:
>
>
> Are you able to isolate what the FH’s are referencing, Sorry that should be RT, not FH. RT == RightTuple, FH == FactHandle.
>>> that might give us an idea of where to look. If you look RightTuple’s they will have a ‘sink' reference (network node) and that sink will have an ‘associates’ map. That map is the rules that node is part of. You can use this to identify the offending rule(s), and maybe paste those rules here?
>>>
>>> Mark
>>> On 30 Mar 2014, at 09:03, kellyajp <<a href="x-msg://133/user/SendEmail.jtp?type=node&node=4029026&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:
>>>
>>>> Thanks
>>>>
>>>> I'll get someone from the team to work on this on Monday. Not sure how easy it will be to isolate but will keep you informed.
>>>>
>>>> Andy
>>>> On 28 Mar 2014, at 23:52, "Mark Proctor [via Drools]" <<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029025&amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&node=4029025&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:
>>>>
>>>>> Can you get us a unit test demonstrating this, and we’ll look into it as a priority.
>>>>>
>>>>> Mark
>>>>> On 28 Mar 2014, at 15:40, kellyajp <<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&node=4029018&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:
>>>>>
>>>>> > We have upgraded from Drools 5 -> Drools 6.01 and when repeating the
>>>>> > processing in each environment we have very different object counts that is
>>>>> > causing memory to run out on Drools 6:
>>>>> >
>>>>> > Below is an extract from a heap dump
>>>>> >
>>>>> > In Drools 6 the main objects in the heap are
>>>>> > 1: 4378006 700480960 org.drools.core.reteoo.RightTuple
>>>>> >
>>>>> > Where as in Drools 5 there are way less objects
>>>>> > 92: 19022 1826112 org.drools.reteoo.RightTuple
>>>>> >
>>>>> > Has anyone else had any experience of this issue?
>>>>> >
>>>>> > Thanks
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > View this message in context: http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010.html
>>>>> > Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
>>>>> > _______________________________________________
>>>>> > rules-dev mailing list
>>>>> > <a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=1">x-msg://131/user/SendEmail.jtp?type=node&node=4029018&i=1" target="_top" rel="nofollow" link="external">[hidden email]
>>>>> > https://lists.jboss.org/mailman/listinfo/rules-dev
>>>>> _______________________________________________
>>>>> rules-dev mailing list
>>>>> <a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=2">x-msg://131/user/SendEmail.jtp?type=node&node=4029018&i=2" target="_top" rel="nofollow" link="external">[hidden email]
>>>>> https://lists.jboss.org/mailman/listinfo/rules-dev
>>>>>
>>>>>
>>>>> If you reply to this email, your message will be added to the discussion below:
>>>>> http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029018.html
>>>>> To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
>>>>> NAML
>>>>
>>>> View this message in context: Re: Possible memory leak in Drools 6.0.1
>>>> Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
>>>> _______________________________________________
>>>> rules-dev mailing list
>>>> <a href="x-msg://133/user/SendEmail.jtp?type=node&node=4029026&i=1" target="_top" rel="nofollow" link="external">[hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/rules-dev
>>>
>>>
>>> _______________________________________________
>>> rules-dev mailing list
>>> <a href="x-msg://133/user/SendEmail.jtp?type=node&node=4029026&i=2" target="_top" rel="nofollow" link="external">[hidden email]
>>> https://lists.jboss.org/mailman/listinfo/rules-dev
>>>
>>> If you reply to this email, your message will be added to the discussion below:
>>> http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029026.html
>>> To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
>>> NAML
>>
>> View this message in context: Re: Possible memory leak in Drools 6.0.1
>> Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
>> _______________________________________________
>> rules-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/rules-dev
>
>
>
> _______________________________________________
> rules-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-dev
>
> If you reply to this email, your message will be added to the discussion below:
> http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029052.html
> To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
> NAML
>
>
>
> NCCF_BASELINE.xls.drl (9M) Download Attachment
>
> View this message in context: Re: Possible memory leak in Drools 6.0.1
> Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
> _______________________________________________
> rules-dev mailing list
> rules-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-dev/attachments/20140402/5a13e2d0/attachment-0001.html
More information about the rules-dev
mailing list