OK, I understand more.
Why not split your rules into two; one that matches all Booking Rules for a transaction
and then another (higher salience) that logs one for only one match?
rule "bootstrap"
salience 0
when
$md:MarketDealModel()
then
insert new MarketDealClassification($md);
end
rule "Find level 1 booking rules"
salience 1
when
dealModel : MarketDealModel( $dealPortfolio : portfolio, $dealTrader : trader,
$dealProduct : product)
productRelatedIndexes : ProductRelatedIndexes( product.internalCode ==
$dealProduct.internalCode, $dealProductIndexes : relatedIndexes)
$mdc : MarketDealClassification(deal == dealModel)
$mb : BTExecutionBookingRuleModel(
priority == BTExecutionBookingRuleModel.LEVEL_1
&& eval(matchTraderCriterion(traderCriterion, $dealTrader))
&& eval(matchPortfolioCriterion(portfolioCriterion, $dealPortfolio))
&& eval(matchIndexCriterion(listedIndexCriterion, $dealProductIndexes))
&& eval(matchProductTypeCriterion(productTypeStringCriterion,
$dealProduct))
)
then
$mdc.addBookingModel($mb);
update($mdc);
end
rule "Level 1 singular matches"
salience 2
when
$mdc : MarketDealClassification( hasSingleMatch == false )
then
retract($mdc);
end
Class MarketDealClassification {
private ArrayList<BTExecutionBookingRuleModel> bookingRules = new
ArrayList<BTExecutionBookingRuleModel>();
private boolean classified = false;
private MarketDealModel deal;
public MarketDealClassification(MarketDealModel deal) {
this.deal=deal;
}
public void addBookingModel(BTExecutionBookingRuleModel bookingRule) {
this.bookingRules.add(bookingRule);
this.hasSingleMatch = (this.bookingRules.length()==1 ? true : false);
}
public boolean hasSingleMatch() {
return this.hasSingleMatch;
}
}
Best add a disclaimer; I've not used Drools for a while and this could turn out to be
complete tripe!
(Preparing to be ripped to shreds by others with more *recent* experience)
Cheers,
Mike
________________________________
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On
Behalf Of Costigliola Joel (EXT)
Sent: 11 September 2009 10:07
To: 'Rules Users List'
Subject: Re: [rules-users] Memory error when inserting facts instatefulsession
Hi Mike,
Quick reply, thanks !
I think I can't use « exists » because I want one and only one booking rule of a
specific level (LEVEL_1, LEVEL_2, ...) matching a deal, that's why the
bookingRuleModels list collected should only contain 1 booking rule. This explain why in
the RHS I'm getting the first booking rule (bookingRuleModels.get(0)).
If there is more than one booking rule (or none), I switch to the second Drools rule
which tries to find a unique LEVEL_2 booking rule, then same thing if none or too many,
switch to LEVEL_3 booking rule ...
Hope you get the idea.
The matchXXX function compare the XXX criterion of a booking rule (ex Trader) with the
corresponding Deal attribute (ex Trader).
Here's the implementation for comparing trader criterion, note that if a booking rule
does not specify a criterion it is considered as matched.
private static boolean matchCriterion(NorthIdEntity<?> ruleCriterionModel,
NorthIdEntity<?> model) {
if (ruleCriterionModel == null) {
// no criterion => matching ok
return true;
}
return ruleCriterionModel.equals(model);
}
I have read that eval is not performant but I thought it was only when executing rules
not at facts insertion phase.
Is this coorect or is eval impacting negatively facts insertion ?
Another idea comes to my mind : can I use a stateless session since once a deal is
classified/booked we don't want to process it again ?
To finish I forgot to give informations on my environment, it may be useful :
- Drools version 5.01
- java 6
- launching my server with following memory options : -Xms256m -Xmx1024m
Thanks again for your thought,
Joel
________________________________
De : rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] De
la part de Anstis, Michael (M.)
Envoyé : vendredi 11 septembre 2009 10:35
À : Rules Users List
Objet : Re: [rules-users] Memory error when inserting facts in stateful session
Hi,
As an aside, you shouldn't need to use a rule flow as the rules themselves determine
which RHS is activated. Could you consider using "exists" instead of the
"ArrayList( size == 1) from collect..." - it looks like you're not too
bothered which Booking Rule matches as (I believe) you have no way to determine which is
at index(0). Also, what do your "matchXXX" functions do? Could you investigate
removing the "evals"?
I don't have any knowledge of your domain - we're all newbies at some time or
other.
With kind regards,
Mike
________________________________
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org]
On Behalf Of Costigliola Joel (EXT)
Sent: 11 September 2009 09:21
To: rules-users(a)lists.jboss.org
Subject: [rules-users] Memory error when inserting facts in stateful session
Hello,
To be short I'm facing some performance/memory problems with Drools which leads to
the error : java.lang.OutOfMemoryError: Java heap space.
That was for the short story, let me now give you more details.
First, I'm a Drools newbie so I certainly have made some "bad" choice.
I'm using Drools 5.01 to classify automatically the deals made by the traders of my
company (this is the functionnal problem I want to solve with Drools).
I have written 4 rules, a rule-flow and start the deals classifying process with a
stateful session.
I have successfully (unit) tested different scenarios, so everything is ok on a
functionnal point of view.
Problems arise when I started to insert more deals in the session, which leads to an
OutOfMemoryError before the call to ksession.fireAllRules. logs are below (after the
drools rule).
Number of facts I have tried to insert in my statefull session :
- 222 booking rule (POJO expressing classification criteria)
- 750 product index
- 750 deals
What drools does here is to find the correct booking rule to apply for each deal (we
need some product index for that).
Can you tell me if those numbers seems unrealistic ?
What can I do to avoid the memory errors ?
To be complete, I show you the drools rules :
rule "Find and apply level 1 booking rule"
lock-on-active true
ruleflow-group "Level 1 booking rule group"
when
dealModel : MarketDealModel( $dealPortfolio : portfolio, $dealTrader :
trader, $dealProduct : product)
// retrieve the ProductRelatedIndexes corresponding to the deal product
(only one by product)
productRelatedIndexes : ProductRelatedIndexes( product.internalCode ==
$dealProduct.internalCode, $dealProductIndexes : relatedIndexes)
// try to find one and only one level 1 matching rule (level 1 <=> all
matching criteria are defined).
bookingRuleModels : ArrayList( size == 1 ) from collect (
BTExecutionBookingRuleModel(
priority == BTExecutionBookingRuleModel.LEVEL_1
&& eval(matchTraderCriterion(traderCriterion,
$dealTrader))
&& eval(matchPortfolioCriterion(portfolioCriterion,
$dealPortfolio))
&& eval(matchIndexCriterion(listedIndexCriterion,
$dealProductIndexes))
&&
eval(matchProductTypeCriterion(productTypeStringCriterion, $dealProduct))
)
)
then
Logger log = LoggerFactory.getLogger("BOOKING RULE ENGINE
LOGGER");
// get the unique collected bookingRuleModel.
BTExecutionBookingRuleModel effectiveBookingRuleModel =
(BTExecutionBookingRuleModel) bookingRuleModels.get(0);
// log.info("Found matching level 1 booking rule --> " +
effectiveBookingRuleModel);
effectiveBookingRuleModel.applyRuleOnDeal(dealModel);
retract( dealModel ); // only needed in use of stateful session to avoid
processing this deal again.
end
I don't put the 3 others, they are basically the same except the priority ==
BTExecutionBookingRuleModel.LEVEL_1 which is done against LEVEL_2, LEVEL_3 and LEVEL_4
rule.
Each rule is in his own ruleflow-group, it is very basic, if first drools rule is not
active then we try the second (with compares priority to
BTExecutionBookingRuleModel.LEVEL_2).
I also put some logs showing that inserting deals fact takes longer and longer :
- the 222 booking rule are inserted in 16ms
- the 750 booking rule are inserted in 46ms
- the 750 facts are inserted at a pace of 5 by second, then it starts to deteriorate to
several seconds for one insert to finish with the OutOfMemoryError.
2009-09-11 09:46:50 134 INFO [booking.impl.DealBookingProcessorImpl] 658
MarketDealModel inserted in Drools session
2009-09-11 09:46:50 244 INFO [booking.impl.DealBookingProcessorImpl] 659
MarketDealModel inserted in Drools session
2009-09-11 09:46:50 369 INFO [booking.impl.DealBookingProcessorImpl] 660
MarketDealModel inserted in Drools session
2009-09-11 09:46:51 197 INFO [booking.impl.DealBookingProcessorImpl] 661
MarketDealModel inserted in Drools session
2009-09-11 09:46:51 306 INFO [booking.impl.DealBookingProcessorImpl] 662
MarketDealModel inserted in Drools session
2009-09-11 09:46:51 540 INFO [booking.impl.DealBookingProcessorImpl] 663
MarketDealModel inserted in Drools session
2009-09-11 09:46:51 650 INFO [booking.impl.DealBookingProcessorImpl] 664
MarketDealModel inserted in Drools session
2009-09-11 09:46:51 775 INFO [booking.impl.DealBookingProcessorImpl] 665
MarketDealModel inserted in Drools session
2009-09-11 09:46:51 884 INFO [booking.impl.DealBookingProcessorImpl] 666
MarketDealModel inserted in Drools session
2009-09-11 09:46:52 009 INFO [booking.impl.DealBookingProcessorImpl] 667
MarketDealModel inserted in Drools session
2009-09-11 09:46:52 134 INFO [booking.impl.DealBookingProcessorImpl] 668
MarketDealModel inserted in Drools session
2009-09-11 09:46:52 243 INFO [booking.impl.DealBookingProcessorImpl] 669
MarketDealModel inserted in Drools session
2009-09-11 09:46:52 368 INFO [booking.impl.DealBookingProcessorImpl] 670
MarketDealModel inserted in Drools session
2009-09-11 09:47:32 784 INFO [booking.impl.DealBookingProcessorImpl] 671
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 003 INFO [booking.impl.DealBookingProcessorImpl] 672
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 128 INFO [booking.impl.DealBookingProcessorImpl] 673
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 253 INFO [booking.impl.DealBookingProcessorImpl] 674
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 362 INFO [booking.impl.DealBookingProcessorImpl] 675
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 487 INFO [booking.impl.DealBookingProcessorImpl] 676
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 596 INFO [booking.impl.DealBookingProcessorImpl] 677
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 721 INFO [booking.impl.DealBookingProcessorImpl] 678
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 831 INFO [booking.impl.DealBookingProcessorImpl] 679
MarketDealModel inserted in Drools session
2009-09-11 09:47:33 956 INFO [booking.impl.DealBookingProcessorImpl] 680
MarketDealModel inserted in Drools session
2009-09-11 09:47:34 128 INFO [booking.impl.DealBookingProcessorImpl] 681
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 218 INFO [booking.impl.DealBookingProcessorImpl] 682
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 374 INFO [booking.impl.DealBookingProcessorImpl] 683
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 483 INFO [booking.impl.DealBookingProcessorImpl] 684
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 608 INFO [booking.impl.DealBookingProcessorImpl] 685
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 733 INFO [booking.impl.DealBookingProcessorImpl] 686
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 858 INFO [booking.impl.DealBookingProcessorImpl] 687
MarketDealModel inserted in Drools session
2009-09-11 09:47:56 968 INFO [booking.impl.DealBookingProcessorImpl] 688
MarketDealModel inserted in Drools session
2009-09-11 09:47:57 093 INFO [booking.impl.DealBookingProcessorImpl] 689
MarketDealModel inserted in Drools session
2009-09-11 09:47:57 218 INFO [booking.impl.DealBookingProcessorImpl] 690
MarketDealModel inserted in Drools session
2009-09-11 09:47:57 421 INFO [booking.impl.DealBookingProcessorImpl] 691
MarketDealModel inserted in Drools session
2009-09-11 09:48:05 404 INFO [booking.impl.DealBookingProcessorImpl] 692
MarketDealModel inserted in Drools session
2009-09-11 09:48:05 529 INFO [booking.impl.DealBookingProcessorImpl] 693
MarketDealModel inserted in Drools session
2009-09-11 09:48:05 654 INFO [booking.impl.DealBookingProcessorImpl] 694
MarketDealModel inserted in Drools session
2009-09-11 09:48:05 763 INFO [booking.impl.DealBookingProcessorImpl] 695
MarketDealModel inserted in Drools session
2009-09-11 09:48:05 888 INFO [booking.impl.DealBookingProcessorImpl] 696
MarketDealModel inserted in Drools session
2009-09-11 09:48:05 998 INFO [booking.impl.DealBookingProcessorImpl] 697
MarketDealModel inserted in Drools session
2009-09-11 09:48:06 123 INFO [booking.impl.DealBookingProcessorImpl] 698
MarketDealModel inserted in Drools session
2009-09-11 09:48:06 357 INFO [booking.impl.DealBookingProcessorImpl] 699
MarketDealModel inserted in Drools session
2009-09-11 09:48:14 184 INFO [booking.impl.DealBookingProcessorImpl] 700
MarketDealModel inserted in Drools session
2009-09-11 09:48:14 293 INFO [booking.impl.DealBookingProcessorImpl] 701
MarketDealModel inserted in Drools session
2009-09-11 09:48:14 418 INFO [booking.impl.DealBookingProcessorImpl] 702
MarketDealModel inserted in Drools session
2009-09-11 09:48:14 543 INFO [booking.impl.DealBookingProcessorImpl] 703
MarketDealModel inserted in Drools session
2009-09-11 09:48:14 668 INFO [booking.impl.DealBookingProcessorImpl] 704
MarketDealModel inserted in Drools session
2009-09-11 09:48:14 856 INFO [booking.impl.DealBookingProcessorImpl] 705
MarketDealModel inserted in Drools session
2009-09-11 09:48:22 761 INFO [booking.impl.DealBookingProcessorImpl] 706
MarketDealModel inserted in Drools session
2009-09-11 09:48:22 886 INFO [booking.impl.DealBookingProcessorImpl] 707
MarketDealModel inserted in Drools session
2009-09-11 09:48:22 995 INFO [booking.impl.DealBookingProcessorImpl] 708
MarketDealModel inserted in Drools session
2009-09-11 09:48:23 120 INFO [booking.impl.DealBookingProcessorImpl] 709
MarketDealModel inserted in Drools session
2009-09-11 09:48:23 323 INFO [booking.impl.DealBookingProcessorImpl] 710
MarketDealModel inserted in Drools session
2009-09-11 09:48:32 166 INFO [booking.impl.DealBookingProcessorImpl] 711
MarketDealModel inserted in Drools session
2009-09-11 09:48:32 290 INFO [booking.impl.DealBookingProcessorImpl] 712
MarketDealModel inserted in Drools session
2009-09-11 09:48:32 400 INFO [booking.impl.DealBookingProcessorImpl] 713
MarketDealModel inserted in Drools session
2009-09-11 09:48:32 634 INFO [booking.impl.DealBookingProcessorImpl] 714
MarketDealModel inserted in Drools session
2009-09-11 09:48:40 570 INFO [booking.impl.DealBookingProcessorImpl] 715
MarketDealModel inserted in Drools session
2009-09-11 09:48:40 695 INFO [booking.impl.DealBookingProcessorImpl] 716
MarketDealModel inserted in Drools session
2009-09-11 09:48:40 899 INFO [booking.impl.DealBookingProcessorImpl] 717
MarketDealModel inserted in Drools session
2009-09-11 09:48:48 850 INFO [booking.impl.DealBookingProcessorImpl] 718
MarketDealModel inserted in Drools session
2009-09-11 09:48:48 975 INFO [booking.impl.DealBookingProcessorImpl] 719
MarketDealModel inserted in Drools session
2009-09-11 09:48:49 272 INFO [booking.impl.DealBookingProcessorImpl] 720
MarketDealModel inserted in Drools session
2009-09-11 09:48:57 209 INFO [booking.impl.DealBookingProcessorImpl] 721
MarketDealModel inserted in Drools session
2009-09-11 09:48:57 505 INFO [booking.impl.DealBookingProcessorImpl] 722
MarketDealModel inserted in Drools session
2009-09-11 09:49:05 598 INFO [booking.impl.DealBookingProcessorImpl] 723
MarketDealModel inserted in Drools session
2009-09-11 09:49:13 722 INFO [booking.impl.DealBookingProcessorImpl] 724
MarketDealModel inserted in Drools session
2009-09-11 09:49:21 752 INFO [booking.impl.DealBookingProcessorImpl] 725
MarketDealModel inserted in Drools session
2009-09-11 09:49:29 813 INFO [booking.impl.DealBookingProcessorImpl] 726
MarketDealModel inserted in Drools session
2009-09-11 09:49:37 921 INFO [booking.impl.DealBookingProcessorImpl] 727
MarketDealModel inserted in Drools session
2009-09-11 09:49:53 809 INFO [booking.impl.DealBookingProcessorImpl] 728
MarketDealModel inserted in Drools session
2009-09-11 09:50:26 507 INFO [booking.impl.DealBookingProcessorImpl] 729
MarketDealModel inserted in Drools session
Exception in thread "Timer-0" java.lang.OutOfMemoryError: Java heap space
at java.lang.Object.clone(Native Method)
at java.util.LinkedList.clone(LinkedList.java:830)
at
com.mchange.v2.resourcepool.BasicResourcePool.cloneOfUnused(BasicResourcePool.java:1661)
at
com.mchange.v2.resourcepool.BasicResourcePool.cullExpired(BasicResourcePool.java:1450)
at
com.mchange.v2.resourcepool.BasicResourcePool.access$1900(BasicResourcePool.java:32)
at
com.mchange.v2.resourcepool.BasicResourcePool$CullTask.run(BasicResourcePool.java:1937)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
Exception in thread "Ice.ThreadPool.Server-1" java.lang.OutOfMemoryError: Java
heap space
at
org.mvel2.integration.impl.ClassImportResolverFactory.<init>(ClassImportResolverFactory.java:49)
at org.mvel2.compiler.CompiledExpression.getValue(CompiledExpression.java:104)
at org.mvel2.MVEL.executeExpression(MVEL.java:978)
at
org.drools.base.mvel.MVELPredicateExpression.evaluate(MVELPredicateExpression.java:75)
at
org.drools.rule.PredicateConstraint.isAllowedCachedLeft(PredicateConstraint.java:295)
at
org.drools.common.SingleBetaConstraints.isAllowedCachedLeft(SingleBetaConstraints.java:138)
at org.drools.reteoo.JoinNode.assertLeftTuple(JoinNode.java:114)
at
org.drools.reteoo.CompositeLeftTupleSinkAdapter.doPropagateAssertLeftTuple(CompositeLeftTupleSinkAdapter.java:145)
at
org.drools.reteoo.CompositeLeftTupleSinkAdapter.createAndPropagateAssertLeftTuple(CompositeLeftTupleSinkAdapter.java:57)
at
org.drools.reteoo.LeftInputAdapterNode.assertObject(LeftInputAdapterNode.java:142)
at
org.drools.reteoo.SingleObjectSinkAdapter.propagateAssertObject(SingleObjectSinkAdapter.java:42)
at org.drools.reteoo.ObjectTypeNode.assertObject(ObjectTypeNode.java:185)
at org.drools.reteoo.EntryPointNode.assertObject(EntryPointNode.java:146)
at
org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:1046)
at
org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:1001)
at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:788)
at
org.drools.impl.StatefulKnowledgeSessionImpl.insert(StatefulKnowledgeSessionImpl.java:216)
at
north.stardust2.services.trading.booking.impl.DealBookingProcessorImpl.insertDealFacts(DealBookingProcessorImpl.java:200)
So to summarize my questions :
- how can I fix the facts insertion ?
- is there some newbie mistakes in my approach ?
Thanks in advance for your help,
Regards,
Joël Costigliola
________________________________
Ce courriel et toutes les pièces jointes sont confidentiels et peuvent être couverts par
un privilège ou une protection légale. Il est établi à l'attention exclusive de ses
destinataires. Toute utilisation de ce courriel non conforme à sa destination, toute
diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation
expresse préalable. Toutes opinions exprimées dans ce courriel ne sauraient nécessairement
refléter celle de Natixis, de ses filiales. Elles sont aussi susceptibles de modification
sans notification préalable. Si vous recevez ce courriel par erreur, merci de le détruire
et d'en avertir immédiatement l'expéditeur. L'Internet ne permettant pas
d'assurer l'intégrité de ce courriel, Natixis décline toute responsabilité
s'il a été altéré, déformé ou falsifié et chaque destinataire qui utilise ce mode de
communication est supposé en accepter les risques.
This email and any attachment are confidential and may be legally privileged or
otherwise protected from disclosure. It is intended only for the stated addressee(s) and
access to it by any other person(s) is unauthorised. Any use, dissemination or disclosure
not in accordance with its purpose, either in whole or in part, is prohibited without our
prior formal approval. Any opinion expressed in this email may not necessarily reflect the
opinion of Natixis, its affiliates. It may also be subject to change without prior notice.
If you are not an addressee, you must not disclose, copy, circulate or in any other way
use or rely on the information contained in this email. If you have received it in error,
please inform us immediately and delete all copies. The Internet can not guarantee the
integrity of this email therefore Natixis shall not be liable for the email if altered,
changed or falsified and anyone who communicates with us by e-mail is taken to accept
these risks.
________________________________
________________________________
Ce courriel et toutes les pièces jointes sont confidentiels et peuvent être couverts par
un privilège ou une protection légale. Il est établi à l'attention exclusive de ses
destinataires. Toute utilisation de ce courriel non conforme à sa destination, toute
diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation
expresse préalable. Toutes opinions exprimées dans ce courriel ne sauraient nécessairement
refléter celle de Natixis, de ses filiales. Elles sont aussi susceptibles de modification
sans notification préalable. Si vous recevez ce courriel par erreur, merci de le détruire
et d'en avertir immédiatement l'expéditeur. L'Internet ne permettant pas
d'assurer l'intégrité de ce courriel, Natixis décline toute responsabilité
s'il a été altéré, déformé ou falsifié et chaque destinataire qui utilise ce mode de
communication est supposé en accepter les risques.
This email and any attachment are confidential and may be legally privileged or otherwise
protected from disclosure. It is intended only for the stated addressee(s) and access to
it by any other person(s) is unauthorised. Any use, dissemination or disclosure not in
accordance with its purpose, either in whole or in part, is prohibited without our prior
formal approval. Any opinion expressed in this email may not necessarily reflect the
opinion of Natixis, its affiliates. It may also be subject to change without prior notice.
If you are not an addressee, you must not disclose, copy, circulate or in any other way
use or rely on the information contained in this email. If you have received it in error,
please inform us immediately and delete all copies. The Internet can not guarantee the
integrity of this email therefore Natixis shall not be liable for the email if altered,
changed or falsified and anyone who communicates with us by e-mail is taken to accept
these risks.
________________________________