Yes we feed the positions into the StatefulSession one at a time.

Is there a difference in how these rules would be treated in Drools 5 and Drool 6. I've hap dumped both instances and there is a huge difference in the number of RighTuple instances being generated in each. The data and rules are exactly the same in both instances. The only option I have is to roll back to the old drools version if we can't resolve the issue; the problem there is that this version dose not include the Mvel fix to stop the jvm freezing. Do you have any suggestions?

Thanks

From: Mark Proctor [via Drools] <[hidden email]>
To: kellyajp <[hidden email]>
Sent: Wednesday, 2 April 2014, 12:41
Subject: Re: Possible memory leak in Drools 6.0.1

not sure why this would have a memory leak, it all looks quite straight forward joins.

I assume you are using stateless sessions and inserting just a single Position instance? Is there a different number of RT’s after a single Position insert?

A small tip, you don’t need so many patterns. All those constraints can go in a single Pattern.

Mark

On 2 Apr 2014, at 11:20, kellyajp <[hidden email]> wrote:

Morning Mark,

That was a useful exercise!!, from a sampled set it looks like the RightTuple instances are coming from a single rule package (nccf).

This rule package comes from a large decision table of that is maintained by our business users. This results in a large DRL file being generated containing 2371 rules.

Below I have included some of the rules that were referenced in the data structures, I have also attached the generated DRL file.

The exact same file processes in Drools 5 but fails in Drools 6.

31usd_lng_corp_bnd_cr_bbb_usd_over0y_under5y_2
// rule values at A650, header at A644
rule "31usd_lng_corp_bnd_cr_bbb_usd_over0y_under5y_2"
salience 64886
when
p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate)
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+157680000000)
Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
Position(positionDetail.overallCreditQualityStep == 2)
Position(positionDetail.valuationType in ('Valuation','Both'))
Position(positionDetail.riskProductType == 'UNKNOWN')
then
result.add(classification=new ActionParametersImpl('BaselineClassification'));
classification.setBaseline('NCCF');
classification.setLineId("NCCF0100579");
classification.setObject(p);
classification.setPositionId(p.getPositionId());
classification.setDataStatus(p.getDataStatus());
classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
classification.setRulePath(ruleHistory.getActivationPath());
classification.add(a=new ActionParametersImpl( 'BaselineValue'));
a.setValueName( 'Balance');
a.setValue( p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
a.setBaselineType( 'MB');
a.setValue(p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
end

31usd_lng_corp_bnd_cr_bbb_usd_over0y_under10y_2
// rule values at A652, header at A644
rule "31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_2"
salience 64884
when
p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+157680000000)
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
Position(positionDetail.overallCreditQualityStep == 2)
Position(positionDetail.valuationType in ('Valuation','Both'))
Position(positionDetail.riskProductType == 'UNKNOWN')
then
result.add(classification=new ActionParametersImpl('BaselineClassification'));
classification.setBaseline('NCCF');
classification.setLineId("NCCF0100581");
classification.setObject(p);
classification.setPositionId(p.getPositionId());
classification.setDataStatus(p.getDataStatus());
classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
classification.setRulePath(ruleHistory.getActivationPath());
classification.add(a=new ActionParametersImpl( 'BaselineValue'));
a.setValueName( 'Balance');
a.setValue( p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
a.setBaselineType( 'MB');
a.setValue(p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
end

31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_1
// rule values at A651, header at A644
rule "31usd_lng_corp_bnd_cr_bbb_usd_over5y_under10y_1"
salience 64885
when
p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+157680000000)
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "USD")
Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "corp")
Position(positionDetail.instrument != null && positionDetail.instrument.typeOfBond != null && positionDetail.instrument.typeOfBond not in ('convertible','covered' ,'jumbo pfandbriefe'))
Position(positionDetail.overallCreditQualityStep == 2)
Position(positionDetail.valuationType in ('Valuation','Both'))
Position(positionDetail.riskProductType != 'UNKNOWN')
then
result.add(classification=new ActionParametersImpl('BaselineClassification'));
classification.setBaseline('NCCF');
classification.setLineId("NCCF0100581");
classification.setObject(p);
classification.setPositionId(p.getPositionId());
classification.setDataStatus(p.getDataStatus());
classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
classification.setRulePath(ruleHistory.getActivationPath());
classification.add(a=new ActionParametersImpl( 'BaselineValue'));
a.setValueName( 'Balance');
a.setValue( p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
a.setCashflowType( 'SETTLEMENT, Settlement');
a.setList(p.getPositionDetail().getCashFlows());
a.setDefaultDaysToCashFlow (1);
classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
a.setBaselineType( 'CB');
a.setCashflowType( 'Principal, Interest');
a.setList(p.getPositionDetail().getCashFlows());
a.setDefaultDaysToCashFlow (15000);
classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
a.setBaselineType( 'MB');
a.setValue(p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
a.setDate(p.getPositionDetail().getInstrument().getMaturityDate());
a.setDefaultDaysToCashFlow (15000);
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
end

08eur_la_lng_qual_mkt_sec_agncy_dbt_7y_Under_10Y_5
// rule values at A954, header at A921
rule "08eur_la_lng_qual_mtk_sec_agncy_dbt_7Y_under_10Y_5"
salience 64582
when
p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
Position(positionDetail.valuationType in ('Valuation','Both'))
Position(positionDetail.instrument != null && positionDetail.instrument.countryOfDomicile in ('AT','BE','BG','CY','CZ','DE','DK','EE','ES','FI','FR' ,'GB','GR' ,'HU' ,'IE' ,'IS' ,'IT' ,'LI' ,'LT' ,'LU' ,'LV' ,'MT' ,'NL' ,'NO' ,'PL' ,'PT' ,'RO' ,'SE' ,'SI' ,'SK'))
Position(positionDetail.instrument != null && positionDetail.instrument.countryOfIncorporation in ('AT' ,'BE' ,'BG' , 'CY' ,'CZ' ,'DE' , 'DK' ,'EE' ,'ES', 'FI' ,'FR' ,'GB' , 'GR' ,'HU' ,'IE' , 'IS' ,'IT' ,'LI' , 'LT' ,'LU' ,'LV' , 'MT' ,'NL' ,'NO' , 'PL' ,'PT' ,'RO','SE' ,'SI' ,'SK'))
Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "EUR")
Position(positionDetail.instrument != null && positionDetail.instrument.collateralType in ('govt liquid gtd','us govt guarant','u.s. government guarantee'))
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+220752000000)
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+315360000000)
Position(positionDetail.valuationType in ('Valuation','Both'))
Position(positionDetail.riskProductType != 'UNKNOWN')
then
result.add(classification=new ActionParametersImpl('BaselineClassification'));
classification.setBaseline('NCCF');
classification.setLineId("NCCF0100405");
classification.setObject(p);
classification.setPositionId(p.getPositionId());
classification.setDataStatus(p.getDataStatus());
classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
classification.setRulePath(ruleHistory.getActivationPath());
classification.add(a=new ActionParametersImpl( 'BaselineValue'));
a.setValueName( 'Balance');
a.setValue( p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
a.setCashflowType( 'SETTLEMENT, Settlement');
a.setList(p.getPositionDetail().getCashFlows());
a.setDefaultDaysToCashFlow (1);
classification.add(a=new ActionParametersImpl( 'BaselineCashflow'));
a.setBaselineType( 'CB');
a.setCashflowType( 'Principal, Interest');
a.setList(p.getPositionDetail().getCashFlows());
a.setDefaultDaysToCashFlow (15000);
classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
a.setBaselineType( 'MB');
a.setValue(p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
a.setDate(p.getPositionDetail().getInstrument().getMaturityDate());
a.setDefaultDaysToCashFlow (15000);
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
end

18cad_lng_sec_iss_prov_over10Y_under35Y_2
// rule values at A152, header at A138
rule "18cad_lng_sec_iss_prov_over10y_under35y_2"
salience 65384
when
p: Position(positionDetail!=null && positionDetail.valuation != null && positionDetail.valuation.mtmValue >=0)
Position(positionDetail.productHierarchy != null && positionDetail.productHierarchy.rbcProductClass == "securities")
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate >=positionDetail.riskBusinessDate+315360000000)
Position(positionDetail.instrument != null && positionDetail.instrument.maturityDate <positionDetail.riskBusinessDate+1103760000000)
Position(positionDetail.instrument != null && positionDetail.instrument.issuerTicker in ('BRCOL','BcpIPB','ALTA','SCDA'
,'MP','MANPAC','MPTB','ONTW'
,'ONT','ONTB','ONTPAC','Q'
,'QTB','QUEPAC','QW','NBRNS'
,'NS','PRINCE','NF','NFTB'))
Position(positionDetail.Valuation!= null && positionDetail. Valuation.ValuationCurrency == "CAD")
Position(positionDetail.instrument != null && positionDetail.instrument.issuerIndustry == "govt regional")
Position(positionDetail.instrument != null && positionDetail.instrument.marketSectorDescription == "govt")
Position(positionDetail.valuationType in ('Valuation','Both'))
Position(positionDetail.riskProductType == 'UNKNOWN')
then
result.add(classification=new ActionParametersImpl('BaselineClassification'));
classification.setBaseline('NCCF');
classification.setLineId("NCCF0100341");
classification.setObject(p);
classification.setPositionId(p.getPositionId());
classification.setDataStatus(p.getDataStatus());
classification.setCountry(p.getPositionDetail().getBook().getTransit().getLegalEntityCountry());
classification.setRulePath(ruleHistory.getActivationPath());
classification.add(a=new ActionParametersImpl( 'BaselineValue'));
a.setValueName( 'Balance');
a.setValue( p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency( p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
classification.add(a=new ActionParametersImpl( 'BaselineTimeBucket'));
a.setBaselineType( 'MB');
a.setValue(p.getPositionDetail().getValuation().getMtmValue());
a.setCurrency(p.getPositionDetail().getValuation().getValuationCurrency());
a.setNettingType('Security');
a.setAdjustmentType('Fungible Long covers Fungible Short,Fungible Short covers Fungible Long');
a.setBucket('Total');
end

From: Mark Proctor [via Drools] <<a href="x-msg://43/user/SendEmail.jtp?type=node&node=4029078&i=0" target="_top" rel="nofollow" link="external">[hidden email]>
To: kellyajp <<a href="x-msg://43/user/SendEmail.jtp?type=node&node=4029078&i=1" target="_top" rel="nofollow" link="external">[hidden email]>
Sent: Tuesday, 1 April 2014, 14:10
Subject: Re: Possible memory leak in Drools 6.0.1

any news?

Mark
On 30 Mar 2014, at 13:38, Mark Proctor <[hidden email]> wrote:

On 30 Mar 2014, at 13:26, kellyajp <[hidden email]> wrote:

I'll get that to you on Monday

On 30 Mar 2014, at 12:59, "Mark Proctor [via Drools]" <<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029027&amp;i=0">x-msg://133/user/SendEmail.jtp?type=node&node=4029027&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

Are you able to isolate what the FH’s are referencing, Sorry that should be RT, not FH. RT == RightTuple, FH == FactHandle.
that might give us an idea of where to look. If you look RightTuple’s they will have a ‘sink' reference (network node) and that sink will have an ‘associates’ map. That map is the rules that node is part of. You can use this to identify the offending rule(s), and maybe paste those rules here?

Mark
On 30 Mar 2014, at 09:03, kellyajp <<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029026&amp;i=0">x-msg://133/user/SendEmail.jtp?type=node&node=4029026&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

Thanks

I'll get someone from the team to work on this on Monday. Not sure how easy it will be to isolate but will keep you informed.

Andy
On 28 Mar 2014, at 23:52, "Mark Proctor [via Drools]" <<a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029025&amp;amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029025&amp;i=0"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029025&amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&node=4029025&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

Can you get us a unit test demonstrating this, and we’ll look into it as a priority.

Mark
On 28 Mar 2014, at 15:40, kellyajp <<a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=0"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=0">x-msg://131/user/SendEmail.jtp?type=node&node=4029018&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

> We have upgraded from Drools 5 -> Drools 6.01 and when repeating the
> processing in each environment we have very different object counts that is
> causing memory to run out on Drools 6:
>
> Below is an extract from a heap dump
>
> In Drools 6 the main objects in the heap are
> 1: 4378006 700480960 org.drools.core.reteoo.RightTuple
>
> Where as in Drools 5 there are way less objects
> 92: 19022 1826112 org.drools.reteoo.RightTuple
>
> Has anyone else had any experience of this issue?
>
> Thanks
>
>
>
> --
> View this message in context: http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010.html
> Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
> _______________________________________________
> rules-dev mailing list
> <a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=1">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=1"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=1">x-msg://131/user/SendEmail.jtp?type=node&node=4029018&i=1" target="_top" rel="nofollow" link="external">[hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-dev
_______________________________________________
rules-dev mailing list
<a href="<a href="<a href="x-msg://131/user/SendEmail.jtp?type=node&amp;amp;node=4029018&amp;amp;i=2">x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=2"><a href="x-msg://131/user/SendEmail.jtp?type=node&amp;node=4029018&amp;i=2">x-msg://131/user/SendEmail.jtp?type=node&node=4029018&i=2" target="_top" rel="nofollow" link="external">[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029018.html
To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML

View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
_______________________________________________
rules-dev mailing list
<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029026&amp;i=1">x-msg://133/user/SendEmail.jtp?type=node&node=4029026&i=1" target="_top" rel="nofollow" link="external">[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

_______________________________________________
rules-dev mailing list
<a href="<a href="x-msg://133/user/SendEmail.jtp?type=node&amp;node=4029026&amp;i=2">x-msg://133/user/SendEmail.jtp?type=node&node=4029026&i=2" target="_top" rel="nofollow" link="external">[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029026.html

To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML

View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

If you reply to this email, your message will be added to the discussion below:
http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029052.html

To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML

NCCF_BASELINE.xls.drl (9M) Download Attachment

View this message in context: Re: Possible memory leak in Drools 6.0.1
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

_______________________________________________
rules-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-dev

If you reply to this email, your message will be added to the discussion below:

http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp4029010p4029081.html

To unsubscribe from Possible memory leak in Drools 6.0.1, click here.
NAML