Afternoon,

just seen that the attachment that I sent you has been made available via the drools user forum. Please can this be removed ASAP. This is client information that is not to be made public.

Thanks


From: Mark Proctor [via Drools] <[hidden email]>
To: kellyajp <[hidden email]>
Sent: Wednesday, 2 April 2014, 12:41
Subject: Re: Possible memory leak in Drools 6.0.1

not sure why this would have a memory leak, it all looks quite straight forward joins.

I assume you are using stateless sessions and inserting just a single Position instance? Is there a different number of RT’s after a single Position insert?

A small tip, you don’t need so many patterns. All those constraints can go in a single Pattern. 

Mark


On 2 Apr 2014, at 11:20, kellyajp <[hidden email]> wrote:

Morning Mark,

That was a useful exercise!!, from a sampled set it looks like the RightTuple instances are coming from a single rule package (nccf).

This rule package comes from a large decision table of that is maintained by our business users. This results in a large DRL file being generated containing 2371 rules. 

Below I have included some of the rules that were referenced in the data structures, I have also attached the generated DRL file.

The exact same file processes in Drools 5 but fails in Drools 6.

31usd_lng_corp_bnd_cr_bbb_usd_over0y_under5y_2
// rule values at A650, header at A644
rule "31usd_lng_corp_bnd_cr_bbb_usd_over0y_under5y_2"
    salience 64886
    when
        p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
        Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate)
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+157680000000)
        Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
        Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
        Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
        Position(positionDetail.overallCreditQualityStep == 2)
        Position(positionDetail.valuationType in ('Valuation','Both'))
        Position(positionDetail.riskProductType == 'UNKNOWN')
    then
        result.add(classification=new ActionParametersImpl('BaselineClassification'));
        classification.setBaseline('NCCF');
        classification.setLineId("NCCF0100579");
        classification.setObject(p);
        classification.setPositionId(p.getPositionId());
        classification.setDataStatus(p.getDataStatus());
        classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
        classification.setRulePath(ruleHistory.getActivationPath());
        classification.add(a=new ActionParametersImpl( 'BaselineValue'));
        a.setValueName( 'Balance');
        a.setValue( p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
        classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
        a.setBaselineType( 'MB');
        a.setValue(p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
end


31usd_lng_corp_bnd_cr_bbb_usd_over0y_under10y_2
// rule values at A652, header at A644
rule "31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_2"
    salience 64884
    when
        p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
        Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+157680000000)
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
        Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
        Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
        Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
        Position(positionDetail.overallCreditQualityStep == 2)
        Position(positionDetail.valuationType in ('Valuation','Both'))
        Position(positionDetail.riskProductType == 'UNKNOWN')
    then
        result.add(classification=new ActionParametersImpl('BaselineClassification'));
        classification.setBaseline('NCCF');
        classification.setLineId("NCCF0100581");
        classification.setObject(p);
        classification.setPositionId(p.getPositionId());
        classification.setDataStatus(p.getDataStatus());
        classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
        classification.setRulePath(ruleHistory.getActivationPath());
        classification.add(a=new ActionParametersImpl( 'BaselineValue'));
        a.setValueName( 'Balance');
        a.setValue( p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
        classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
        a.setBaselineType( 'MB');
        a.setValue(p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
end


31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_1
// rule values at A651, header at A644
rule "31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_1"
    salience 64885
    when
        p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
        Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+157680000000)
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
        Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
        Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
        Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
        Position(positionDetail.overallCreditQualityStep == 2)
        Position(positionDetail.valuationType in ('Valuation','Both'))
        Position(positionDetail.riskProductType != 'UNKNOWN')
    then
        result.add(classification=new ActionParametersImpl('BaselineClassification'));
        classification.setBaseline('NCCF');
        classification.setLineId("NCCF0100581");
        classification.setObject(p);
        classification.setPositionId(p.getPositionId());
        classification.setDataStatus(p.getDataStatus());
        classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
        classification.setRulePath(ruleHistory.getActivationPath());
        classification.add(a=new ActionParametersImpl( 'BaselineValue'));
        a.setValueName( 'Balance');
        a.setValue( p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
        classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
        a.setCashflowType( 'SETTLEMENT, Settlement');
        a.setList(p.getPositionDetail().getCashFlows());
        a.setDefaultDaysToCashFlow (1);
        classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
        a.setBaselineType( 'CB');
        a.setCashflowType( 'Principal, Interest');
        a.setList(p.getPositionDetail().getCashFlows());
        a.setDefaultDaysToCashFlow (15000);
        classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
        a.setBaselineType( 'MB');
        a.setValue(p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
        a.setDate(p.getPositionDetail().getInstrument().getMaturityDate());
        a.setDefaultDaysToCashFlow (15000);
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
end


08eur_la_lng_qual_mkt_sec_agncy_dbt_7y_Under_10Y_5
// rule values at A954, header at A921
rule "08eur_la_lng_qual_mtk_sec_agncy_dbt_7Y_under_10Y_5"
    salience 64582
    when
        p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
        Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
        Position(positionDetail.valuationType in ('Valuation','Both'))
        Position(positionDetail.instrument != null && positionDetail.instrument.countryOfDomicile in ('AT','BE','BG','CY','CZ','DE','DK','EE','ES','FI','FR' ,'GB','GR' ,'HU' ,'IE' ,'IS' ,'IT' ,'LI' ,'LT' ,'LU' ,'LV' ,'MT' ,'NL' ,'NO' ,'PL' ,'PT' ,'RO' ,'SE' ,'SI' ,'SK'))
        Position(positionDetail.instrument != null && positionDetail.instrument.countryOfIncorporation in ('AT' ,'BE' ,'BG' , 'CY' ,'CZ' ,'DE'  , 'DK' ,'EE' ,'ES', 'FI' ,'FR' ,'GB' , 'GR' ,'HU' ,'IE' , 'IS' ,'IT' ,'LI' , 'LT' ,'LU' ,'LV' , 'MT' ,'NL' ,'NO' , 'PL' ,'PT' ,'RO','SE' ,'SI' ,'SK'))
        Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "EUR")
        Position(positionDetail.instrument != null && positionDetail.instrument.collateralType in ('govt liquid gtd','us govt guarant','u.s. government guarantee'))
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+220752000000)
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
        Position(positionDetail.valuationType in ('Valuation','Both'))
        Position(positionDetail.riskProductType != 'UNKNOWN')
    then
        result.add(classification=new ActionParametersImpl('BaselineClassification'));
        classification.setBaseline('NCCF');
        classification.setLineId("NCCF0100405");
        classification.setObject(p);
        classification.setPositionId(p.getPositionId());
        classification.setDataStatus(p.getDataStatus());
        classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
        classification.setRulePath(ruleHistory.getActivationPath());
        classification.add(a=new ActionParametersImpl( 'BaselineValue'));
        a.setValueName( 'Balance');
        a.setValue( p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
        classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
        a.setCashflowType( 'SETTLEMENT, Settlement');
        a.setList(p.getPositionDetail().getCashFlows());
        a.setDefaultDaysToCashFlow (1);
        classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
        a.setBaselineType( 'CB');
        a.setCashflowType( 'Principal, Interest');
        a.setList(p.getPositionDetail().getCashFlows());
        a.setDefaultDaysToCashFlow (15000);
        classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
        a.setBaselineType( 'MB');
        a.setValue(p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
        a.setDate(p.getPositionDetail().getInstrument().getMaturityDate());
        a.setDefaultDaysToCashFlow (15000);
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
end

18cad_lng_sec_iss_prov_over10Y_under35Y_2
// rule values at A152, header at A138
rule "18cad_lng_sec_iss_prov_over10y_under35y_2"
    salience 65384
    when
        p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
        Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+315360000000)
        Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+1103760000000)
        Position(positionDetail.instrument != null && positionDetail.instrument.issuerTicker in ('BRCOL','BcpIPB','ALTA','SCDA'
,'MP','MANPAC','MPTB','ONTW'
,'ONT','ONTB','ONTPAC','Q'
,'QTB','QUEPAC','QW','NBRNS'
,'NS','PRINCE','NF','NFTB'))
        Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "CAD")
        Position(positionDetail.instrument != null && positionDetail.instrument.issuerIndustry == "govt regional")
        Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "govt")
        Position(positionDetail.valuationType in ('Valuation','Both'))
        Position(positionDetail.riskProductType == 'UNKNOWN')
    then
        result.add(classification=new ActionParametersImpl('BaselineClassification'));
        classification.setBaseline('NCCF');
        classification.setLineId("NCCF0100341");
        classification.setObject(p);
        classification.setPositionId(p.getPositionId());
        classification.setDataStatus(p.getDataStatus());
        classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
        classification.setRulePath(ruleHistory.getActivationPath());
        classification.add(a=new ActionParametersImpl( 'BaselineValue'));
        a.setValueName( 'Balance');
        a.setValue( p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
        classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
        a.setBaselineType( 'MB');
        a.setValue(p.getPositionDetail().getValuation().getMtmValue());
        a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
        a.setNettingType('Security');
        a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
        a.setBucket('Total');
end






From: Mark Proctor [via Drools] <<a href="x-msg://43/user/SendEmail.jtp?type=node&amp;node=4029078&amp;i=0" target="_top" rel="nofollow" link="external">[hidden email]>
To: kellyajp <<a href="x-msg://43/user/SendEmail.jtp?type=node&amp;node=4029078&amp;i=1" target="_top" rel="nofollow" link="external">[hidden email]>
Sent: Tuesday, 1 April 2014, 14:10
Subject: Re: Possible memory leak in Drools 6.0.1

any news?

Mark
On 30 Mar 2014, at 13:38, Mark Proctor <[hidden email]> wrote:


On 30 Mar 2014, at 13:26, kellyajp <[hidden email]> wrote:

I'll get that to you on Monday 

On 30 Mar 2014, at 12:59, "Mark Proctor [via Drools]" <<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;amp;node=4029027&amp;amp;i=0">x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029027&amp;i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

Are you able to isolate what the FH’s are referencing, Sorry that should be RT, not FH. RT == RightTuple, FH == FactHandle.
that might give us an idea of where to look. If you look RightTuple’s they will have a ‘sink' reference (network node) and that sink will have an ‘associates’ map. That map is the rules that node is part of. You can use this to identify the offending rule(s), and maybe paste those rules here?

Mark
On 30 Mar 2014, at 09:03, kellyajp <<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;amp;node=4029026&amp;amp;i=0">x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029026&amp;i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

Thanks

I'll get someone from the team to work on this on Monday. Not sure how easy it will be to isolate but will keep you informed.

Andy
On 28 Mar 2014, at 23:52, "Mark Proctor [via Drools]" <<a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;amp;node=4029025&amp;amp;amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029025&amp;amp;i=0"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029025&amp;amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029025&amp;i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

Can you get us a unit test demonstrating this, and we’ll look into it as a priority. 

Mark 
On 28 Mar 2014, at 15:40, kellyajp <<a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;amp;node=4029018&amp;amp;amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=0"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote: 

> We have upgraded from Drools 5 -> Drools 6.01 and when repeating the 
> processing in each environment we have very different object counts that is 
> causing memory to run out on Drools 6: 
> 
> Below is an extract from a heap dump 
> 
> In Drools 6 the main objects in the heap are 
> 1:       4378006      700480960  org.drools.core.reteoo.RightTuple 
> 
> Where as in Drools 5 there are way less objects 
> 92:         19022        1826112  org.drools.reteoo.RightTuple 
> 
> Has anyone else had any experience of this issue? 
> 
> Thanks 
> 
> 
> 
> -- 
> View this message in context: http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010.html
> Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com. 
> _______________________________________________ 
> rules-dev mailing list 
> <a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;amp;node=4029018&amp;amp;amp;i=1">x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=1"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=1">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=1" target="_top" rel="nofollow" link="external">[hidden email] 
> https://lists.jboss.org/mailman/listinfo/rules-dev
_______________________________________________ 
rules-dev mailing list 
<a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;amp;node=4029018&amp;amp;amp;i=2">x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=2"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=2">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=2" target="_top" rel="nofollow" link="external">[hidden email] 
https://lists.jboss.org/mailman/listinfo/rules-dev


If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029018.html
To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML


View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
_______________________________________________
rules-dev mailing list
<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;amp;node=4029026&amp;amp;i=1">x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029026&amp;i=1" target="_top" rel="nofollow" link="external">[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev


_______________________________________________
rules-dev mailing list
<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;amp;node=4029026&amp;amp;i=2">x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029026&amp;i=2" target="_top" rel="nofollow" link="external">[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev


If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029026.html
To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML


View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev



_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev


If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029052.html
To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML



NCCF_BASELINE.xls.drl (9M) Download Attachment


View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev


_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev


If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029081.html
To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML




View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.