Rule performance with accumulate
by Ryan Crumley
Hi,
I am investigating performance of a Drools 5.4 stateful knowledge session.
This session has about 200 rules, 200k facts and takes about 1 hour to run
to completion. Looking at the profile there is a hotspot that consumes
almost 65% of the cpu time: java.util.AbstractList.hashCode().
Here is the full stack:
com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cbDefaultConsequenceInvoker.evaluate(KnowledgeHelper,
WorkingMemory)
com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cbDefaultConsequenceInvokerGenerated.evaluate(KnowledgeHelper,
WorkingMemory)
com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cb.defaultConsequence(KnowledgeHelper,
List, FactHandle, GradingFact, FactHandle, ReportNode, FactHandle,
WeightsHolder, FactHandle, Logger)
org.drools.base.DefaultKnowledgeHelper.update(FactHandle, long)
org.drools.common.NamedEntryPoint.update(FactHandle, Object, long,
Activation)
org.drools.common.NamedEntryPoint.update(FactHandle, Object, long,
Activation)
org.drools.common.PropagationContextImpl.evaluateActionQueue(InternalWorkingMemory)
org.drools.reteoo.ReteooWorkingMemory$EvaluateResultConstraints.execute(InternalWorkingMemory)
org.drools.reteoo.AccumulateNode.evaluateResultConstraints(AccumulateNode$ActivitySource,
LeftTuple, PropagationContext, InternalWorkingMemory,
AccumulateNode$AccumulateMemory, AccumulateNode$AccumulateContext, boolean)
org.drools.common.DefaultFactHandle.setObject(Object)
java.util.AbstractList.hashCode()
I believe the following clues can be extracted:
- "Rule_Set_weights" was fired and a fact was modified (confirmed by
examining the rule definition)
- The fact modification caused the pre-conditions for other rules to be
computed.
- One of these rules has an accumulate condition that accumulates into an
AbstractList.
- This list is very very large. So large that looping through the elements
in the list and aggregating the hashCode of individual elements dominates
execution time (the individual element hashCode doesn't even show up in the
profile… either its very fast or maybe its identify hashCode which the
profiler might filter?).
- Accumulate is either working on a large set of data or the same
accumulate is evaluated many many times.
Is my analysis correct? Are there clues that I am missing?
I have 15 rules that use accumulate… However none accumulate with a result
of List. Most accumulate using sum() and count() (result of Number). A few
use collectSet(). A few more aggregate into a result with a custom type.
A few other notes:
- All accumulate conditions are the last condition in the WHEN clause.
- I use agenda groups to separate fact processing into phases. Rules that
accumulate are in a separate agenda group from rules that modify/insert
facts that are used in accumulation. I hope this prevents the accumulate
condition from being evaluated until all the rules that modify the facts
accumulate needs are done firing. I suspect this may not be working as I
expect. I haven't put together an example to investigate.
- When accumulating into a set, the rule condition looks like this:
$factName : Set() from accumulate( FactMatch( $field : field ),
collectionSet( $field ) )
How can I narrow down this further?
Are there any general rules to follow to optimize use of accumulate in
conditions?
Thanks,
Ryan
13 years, 2 months
guvnor - out of memory
by kurrent93
Hi
I recently starting using Streaming mode / CEP rules, and I'm not sure if
that is the reason, but I am getting guvnor dying due to out of memory.
Besides putting extra ram in the server, can anyone shed some light on this?
Any suggested solutions?
Thanks
03:44:44,152 ERROR
[org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/guvnor].[PackageDeploymentServlet]]
(http--0.0.0.0-8080-2) Servlet.service() for servlet
PackageDeploymentServlet threw exception:
javax.enterprise.event.ObserverException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[rt.jar:1.7.0_09]
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
[rt.jar:1.7.0_09]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[rt.jar:1.7.0_09]
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
[rt.jar:1.7.0_09]
at java.lang.Class.newInstance0(Class.java:372) [rt.jar:1.7.0_09]
at java.lang.Class.newInstance(Class.java:325) [rt.jar:1.7.0_09]
at
org.jboss.weld.util.reflection.SecureReflections$16.work(SecureReflections.java:344)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInstantiation(SecureReflectionAccess.java:173)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.util.reflection.SecureReflections.newInstance(SecureReflections.java:341)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at org.jboss.weld.injection.Exceptions.rethrowException(Exceptions.java:33)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at org.jboss.weld.injection.Exceptions.rethrowException(Exceptions.java:73)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.injection.MethodInjectionPoint.invokeOnInstanceWithSpecialValue(MethodInjectionPoint.java:162)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:241)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:229)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:207)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.manager.BeanManagerImpl.notifyObservers(BeanManagerImpl.java:573)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.manager.BeanManagerImpl.fireEvent(BeanManagerImpl.java:563)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.weld.manager.BeanManagerImpl.fireEvent(BeanManagerImpl.java:558)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.jboss.solder.servlet.exception.CatchExceptionFilter.doFilter(CatchExceptionFilter.java:74)
[solder-impl-3.1.1.Final.jar:3.1.1.Final]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
[jbossweb-7.0.13.Final.jar:]
at
org.jboss.solder.servlet.event.ServletEventBridgeFilter.doFilter(ServletEventBridgeFilter.java:74)
[solder-impl-3.1.1.Final.jar:3.1.1.Final]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
[jbossweb-7.0.13.Final.jar:]
at
org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
[jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
[jbossweb-7.0.13.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
[jbossweb-7.0.13.Final.jar:]
at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
Caused by: javax.servlet.ServletException: Servlet execution threw an
exception
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:346)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
[jbossweb-7.0.13.Final.jar:]
at
org.jboss.weld.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:62)
[weld-core-1.1.6.Final.jar:2012-03-21 18:52]
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
[jbossweb-7.0.13.Final.jar:]
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
[jbossweb-7.0.13.Final.jar:]
at
org.jboss.solder.servlet.exception.CatchExceptionFilter.doFilter(CatchExceptionFilter.java:65)
[solder-impl-3.1.1.Final.jar:3.1.1.Final]
... 16 more
Caused by: java.lang.OutOfMemoryError: Java heap space
--
View this message in context: http://drools.46999.n3.nabble.com/guvnor-out-of-memory-tp4021555.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
13 years, 2 months
New object in XLS ACTION
by aleste
I'm trying to create a new object, populate a number of attributes in it and
insert it in memory as part of the ACTION in a decision table XLS.
I was able to do that in a single XLS column:
ACTION
insertLogical( new RuleError(t, Constants.ErrorType.PROMOTION, $1, $2));
Or using the guided decision tables in Guvnor:
FeeDTO [f3]
amount
10
20
(...)
But I have not been able to do it using an XLS and having multiple ACTION
columns. Something like:
ACTION ACTION
FeeDTO feeDTO = new FeeDTO();
feeDTO.setAmount(new BigDecimal($1)); feeDTO.setReason("$1");
This is the rule that I'd be expecting as result of the XLS compilation:
//from row number: 3
rule "Row 3 FeesTest"
dialect "mvel"
when
f1 : FinancialTransactionDTO( transactionType == TRANSACTION_TYPE.DEPOSIT
)
then
FeeDTO f3 = new FeeDTO();
f3.setAmount( 20 );
f3.setType( "Some Description" );
f3.setOverrideable (true);
insert( f3 );
end
--
View this message in context: http://drools.46999.n3.nabble.com/New-object-in-XLS-ACTION-tp4021374.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
13 years, 2 months
Drools KnowledgeAgent not picking up drl resource updates
by js203
Hi,
I am looking for a way to update drools rules files in our production
environment without having to go through a full application rebuild and
redeploy to pick up .drl modifications.
To make this seamless, I'm investigating using change sets but I can't get a
KnowledgeBase to update when a .drl is updated.
Here is the code I have, all files are in the /root/com/me/drools package:
*1. DroolsConfig.java - Helper class to configure and start a
KnowledgeAgent. It references a local change-set.xml to manage the Drools
resources and scan for resource changes every 2 seconds*
package com.me.drools;
import org.drools.KnowledgeBase;
import org.drools.agent.KnowledgeAgent;
import org.drools.agent.KnowledgeAgentConfiguration;
import org.drools.agent.KnowledgeAgentFactory;
import org.drools.builder.KnowledgeBuilderConfiguration;
import org.drools.io.ResourceChangeScannerConfiguration;
import org.drools.io.ResourceFactory;
public class DroolsConfig {
public KnowledgeBase getAgentKnowledgeBase() {
ResourceFactory.getResourceChangeNotifierService().start();
ResourceFactory.getResourceChangeScannerService().start();
ResourceChangeScannerConfiguration sconf =
ResourceFactory.getResourceChangeScannerService().newResourceChangeScannerConfiguration();
sconf.setProperty("drools.resource.scanner.interval", "2");
ResourceFactory.getResourceChangeScannerService().configure(sconf);
KnowledgeAgentConfiguration aconf =
KnowledgeAgentFactory.newKnowledgeAgentConfiguration();
aconf.setProperty("drools.agent.scanDirectories", "true");
aconf.setProperty("drools.agent.scanResources", "true");
aconf.setProperty("drools.agent.newInstance", "true");
KnowledgeAgent knowledgeAgent =
KnowledgeAgentFactory.newKnowledgeAgent("CS", aconf);
knowledgeAgent.applyChangeSet(ResourceFactory.newFileResource("/root/com/me/drools/change-set.xml"));
return knowledgeAgent.getKnowledgeBase();
}
}
*2. change-set.xml - Referenced from DroolsConfig and manages a local rules
file*
<change-set xmlns='http://drools.org/drools-5.0/change-set'
xmlns:xs='http://www.w3.org/2001/XMLSchema-instance'
xs:schemaLocation='http://drools.org/drools-5.0/change-set.xsd
http://anonsvn.jboss.org/repos/labs/labs/jbossrules/trunk/drools-api/src/...'>
<add>
<resource source='file:/root/com/me/drools/personrules.drl'
type='DRL' />
</add>
</change-set>
To test this out I have the following JUnit. The aim is to run the test and
during the loop execution, change the condition in the drl so that the
KnowledgeBase updates and rule results differ.
*3. DroolsConfigTest.java - Tests DroolsConfig. Rule results should change
during loop execution if drl is updated*
package com.me.drools;
import org.drools.KnowledgeBase;
import org.drools.runtime.StatelessKnowledgeSession;
import org.junit.Test;
public class DroolsConfigTest {
@Test
public void testGetAgentKnowledgeBase() throws Exception {
Person person = new Person();
DroolsConfig droolsConfig = new DroolsConfig();
KnowledgeBase kb = droolsConfig.getAgentKnowledgeBase();
for (int i = 0; i < 10; i++) {
StatelessKnowledgeSession session =
kb.newStatelessKnowledgeSession();
session.execute(person);
Thread.currentThread().sleep(3000);
System.out.println(person.getErrors().size());
}
System.out.println("Errors: " + person.getErrors());
}
}
Just for reference here is PersonTO and the person.drl rules file.
*4. Person.java - Simple object to fire rules against*
package com.me.drools;
import java.util.ArrayList;
import java.util.List;
public class Person {
private String name;
private List<String> errors = new ArrayList<String>();
public void addError(final String error) {
errors.add(error);
}
public List<String> getErrors() {
return errors;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
*5. personrules.drl*
package com.me.drools
import com.me.drools.Person
dialect "java"
rule "name_is_null"
when
$p : Person( name == null )
then
$p.addError("Name is null");
end
So, as mentioned when the for loop in DroolsConfigTest is running, I'm
changing the condition in the personrules.drl file from name == null to name
!= null which should mean that errors are no longer added to the errors List
in Person.class but the error count keeps on going up.
Can anybody see why the KnowledgeAgent isn't updating and how I can make
this configuration work so that I can change the drl on the fly and have the
rules behave accordingly?
Thanks
--
View this message in context: http://drools.46999.n3.nabble.com/Drools-KnowledgeAgent-not-picking-up-dr...
Sent from the Drools: User forum mailing list archive at Nabble.com.
13 years, 2 months
Difficulty determining why I have score corruption
by megwoods
Hi,
I have been at this a couple of days now but I am getting score corruption
from a rule that does the following:
rule "to bigger gap"
when
$res: ResourceInstance()
$resList : ArrayList(size > 1)
from collect( TripAtTimeWithResource(resource == $res) )
then
-- Sort the list using custom comparator.
-- Measure a series of deltas between elements in the list.
-- If the delta is larger than a threshold add a soft constraint.
insertLogical(new IntConstraintOccurrence("to bigger gap",
ConstraintType.NEGATIVE_SOFT,delta * 10, ttrL));
end
The TripAtTimeWithResource is the planning entity.
I have checked the cloning and been through and checked that all equals and
hashcode methods and they are correct.
If I take this rule out the implementation functions as correctly.
Interestingly though I first prototyped this using a simple score and with
the same 'when' and 'then' clause (inserting a different fact) and no score
corruption was observed.
If anyone has any suggestions I would really like to hear from you.
Cheers Megan Woods
Extract from exception:
The workingMemory has 3 ConstraintOccurrence(s) in excess:
to bigger gap/NEGATIVE_SOFT:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@43b, resource=All(17), startTime=TimeUnit
[minute=285]]]=750
to bigger gap/NEGATIVE_SOFT:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@46c, resource=All(17), startTime=TimeUnit
[minute=75]]]=1700
to bigger gap/NEGATIVE_SOFT:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@421, resource=All(17), startTime=TimeUnit
[minute=75]]]=960
The workingMemory has 4 ConstraintOccurrence(s) lacking:
overlapping trips with same
resource/NEGATIVE_HARD:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@43b, resource=All(17), startTime=TimeUnit
[minute=285]]]=100
to bigger gap/NEGATIVE_SOFT:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@421, resource=All(17), startTime=TimeUnit
[minute=75]]]=1710
to bigger gap/NEGATIVE_SOFT:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@43b, resource=All(17), startTime=TimeUnit
[minute=285]]]=1650
to bigger gap/NEGATIVE_SOFT:[TripAtTimeWithResource
[trip=a.c.t.p.p.m.TripInstance@46c, resource=All(17), startTime=TimeUnit
[minute=75]]]=950
--
View this message in context: http://drools.46999.n3.nabble.com/Difficulty-determining-why-I-have-score...
Sent from the Drools: User forum mailing list archive at Nabble.com.
13 years, 2 months
[Switchyard RuleService] Rule doesnt fire when in FireAllRules mode
by kurrent93
Im confused about why my rule isnt firing.
Im using Jboss SwitchYard.
This is the only difference from the switchyard perspective:
<rules:action name="sendData" type="FIRE_ALL_RULES"
entryPoint="AccountData"/>
And when I use EXECUTE the rule fires.
When I use FIRE_ALL_RULES, it never gets activated.
This is my rule:
| rule "OpenTrade-Open"
2. | agenda-group "OpenTrade"
3. | no-loop true
4. | dialect "mvel"
5. | when
6. | Tick( ) over window:time (1m) from entry-point "AccountData"
7. | then
8. | TradeOrder newTrade = new TradeOrder();
9. | newTrade.setSymbol( "EURUSD" );
10. | newTrade.setVolume( 1 );
11. | newTrade.setMt4username( "4002642" );
12. | newTrade.setStatus( OrderStatus.OPENING );
13. | newTrade.setOperation( "OP_BUY" );
14. | newTrade.setComment( "From Magick.Rhythm" );
15. | insert( newTrade );
16. | modifiedOrderList.add(newTrade);
17. | end
--
View this message in context: http://drools.46999.n3.nabble.com/Switchyard-RuleService-Rule-doesnt-fire...
Sent from the Drools: User forum mailing list archive at Nabble.com.
13 years, 2 months
