Multiple threads compiling rules is slow
by RichardAmbridge
I am trying to improve the performance of compiling our rules into packages.
I started to making the process into threads. The build server is multicore
so multi-threads should help.
However, I found it was slower in threads compared to a single compile..
So I did some digging.. (Drools 5.5.0)
Below is a sample java application I created.
What it does is create a StringBuilder with 30000 rules in it.
Then it compiles the rules.
Next, it creates 4 StringBuilders, putting 7500 rules in each, and compiles
each of them in a thread.
The results are:
Single Thread:
Starting build size=2508946
Time to compile:16963 ms
Multi Thread:
Thread [3] compile time=15455 ms
Thread [2] compile time=15580 ms
Thread [0] compile time=15633 ms
Thread [1] compile time=15674 ms
Time to compile:16075 ms
So, the single compile takes 16 seconds.. The 4 threads take 16 seconds..
If I watch the process, only 1 thread is running at a time, all the other
threads are blocked..
with the following:
Name: Thread-3
State: BLOCKED on sun.misc.Launcher$AppClassLoader@138d107f owned by:
Thread-1
Total blocked: 16,135 Total waited: 0
Stack trace:
java.lang.Class.forName0(Native Method)
java.lang.Class.forName(Class.java:249)
org.drools.util.CompositeClassLoader$CachingLoader.load(CompositeClassLoader.java:258)
org.drools.util.CompositeClassLoader$CachingLoader.load(CompositeClassLoader.java:237)
org.drools.util.CompositeClassLoader.loadClass(CompositeClassLoader.java:88)
java.lang.ClassLoader.loadClass(ClassLoader.java:247)
java.lang.Class.forName0(Native Method)
java.lang.Class.forName(Class.java:249)
org.mvel2.optimizers.AbstractOptimizer.forNameWithInner(AbstractOptimizer.java:166)
org.mvel2.optimizers.AbstractOptimizer.tryStaticAccess(AbstractOptimizer.java:79)
org.mvel2.compiler.PropertyVerifier.getBeanProperty(PropertyVerifier.java:257)
org.mvel2.compiler.PropertyVerifier.analyze(PropertyVerifier.java:117)
org.mvel2.compiler.ExpressionCompiler.verify(ExpressionCompiler.java:407)
org.mvel2.compiler.ExpressionCompiler._compile(ExpressionCompiler.java:273)
org.mvel2.compiler.ExpressionCompiler.compile(ExpressionCompiler.java:62)
org.mvel2.MVEL.analyze(MVEL.java:680)
org.mvel2.MVEL.analyze(MVEL.java:685)
org.drools.rule.builder.dialect.mvel.MVELExprAnalyzer.analyzeExpression(MVELExprAnalyzer.java:114)
org.drools.rule.builder.dialect.mvel.MVELDialect.analyzeBlock(MVELDialect.java:535)
org.drools.rule.builder.dialect.mvel.MVELConsequenceBuilder.build(MVELConsequenceBuilder.java:116)
org.drools.rule.builder.RuleBuilder.build(RuleBuilder.java:103)
org.drools.compiler.PackageBuilder.addRule(PackageBuilder.java:2830)
org.drools.compiler.PackageBuilder.compileRules(PackageBuilder.java:970)
org.drools.compiler.PackageBuilder.compileAllRules(PackageBuilder.java:879)
org.drools.compiler.PackageBuilder.addPackage(PackageBuilder.java:871)
org.drools.compiler.PackageBuilder.addPackageFromDrl(PackageBuilder.java:440)
org.drools.compiler.PackageBuilder.addPackageFromDrl(PackageBuilder.java:418)
droolscompilermultithread.PkgThread.run(PkgThread.java:23)
Can anyone assist on why only 1 thread blocks all the others?
Thanks
///begin code
package droolscompilermultithread;
import java.io.IOException;
import java.io.StringReader;
import org.drools.compiler.DroolsParserException;
import org.drools.compiler.PackageBuilder;
public class Main {
private static int todo = 30000;
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws Exception {
Main m = new Main();
m.doSingle();
m.doThread(4);
}
private void addHead(StringBuilder sb) {
sb.append("package droolscompilermultithread\n");
sb.append("import Main.MyObject;\n");
}
private void addRule(StringBuilder sb, int id) {
sb.append("rule myRule").append(id).append("\n");
sb.append("dialect \"mvel\"\n");
sb.append("when\n");
sb.append(" $m : MyObject(s1==\"a\")\n");
sb.append("then\n");
sb.append(" $m.s1=\"b\";");
sb.append("end\n\n");
}
/**
* Create a single buffer and compile it all in one thread
* @throws DroolsParserException
* @throws IOException
*/
private void doSingle() throws DroolsParserException, IOException {
StringBuilder sb = new StringBuilder();
addHead(sb);
int count = todo;
while (count-- > 0) {
addRule(sb, count);
}
System.out.println("Starting build size=" + sb.length());
long start = System.currentTimeMillis();
PackageBuilder pkgB = new PackageBuilder();
pkgB.addPackageFromDrl(new StringReader(sb.toString()));
if (pkgB.hasErrors()) {
System.err.println("Errors:" + pkgB.getErrors().toString());
}
long end = System.currentTimeMillis();
System.out.println("Time to compile:" + (end - start) + " ms");
}
/**
* Create X threads, and in each create a StringBuilder which contains
todo/thread rules
* Then in Threads build each of those packages
* @param threads
* @throws DroolsParserException
* @throws IOException
*/
private void doThread(int threads) throws DroolsParserException,
IOException {
StringBuilder[] sbs = new StringBuilder[threads];
for (int t = 0; t < threads; t++) {
sbs[t] = new StringBuilder();
addHead(sbs[t]);
}
int count = todo;
int threadpos = 0;
while (count-- > 0) {
addRule(sbs[threadpos++], count);
if (threadpos == threads) {
threadpos = 0;
}
}
long start = System.currentTimeMillis();
PkgThread[] thrds = new PkgThread[threads];
for (int t = 0; t < threads; t++) {
thrds[t] = new PkgThread(sbs[t], t);
thrds[t].start();
}
for (int t = 0; t < threads; t++) {
while (thrds[t].isAlive()) {
//Wait for the thread to complete
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
}
}
if (thrds[t].pkgB.hasErrors()) {
System.err.println(thrds[t].pkgB.getErrors().toString());
}
}
long end = System.currentTimeMillis();
System.out.println("Time to compile:" + (end - start) + " ms");
}
/**
* Object used in drools
*/
public class MyObject {
int a;
String s1;
public String getS1() {
return s1;
}
public void setS1(String s1) {
this.s1 = s1;
}
public int getA() {
return a;
}
public void setA(int a) {
this.a = a;
}
}
/**
* A simple thread that takes a stringbuilder and PackageBuilder it
*/
class PkgThread extends Thread {
StringBuilder sb;
int id;
boolean exception = false;
PackageBuilder pkgB = new PackageBuilder();
long took;
public PkgThread(StringBuilder sb, int id) {
this.sb = sb;
this.id = id;
}
@Override
public void run() {
try {
long start = System.currentTimeMillis();
pkgB.addPackageFromDrl(new StringReader(sb.toString()));
long end = System.currentTimeMillis();
took=end-start;
System.out.println("Thread [" + id + "] compile time=" +
took + " ms");
} catch (Exception ex) {
exception = true;
}
}
public long getTook() {
return took;
}
public boolean isException() {
return exception;
}
public PackageBuilder getPkgB() {
return pkgB;
}
}
/*
*
*
*/
}
//end code
--
View this message in context: http://drools.46999.n3.nabble.com/Multiple-threads-compiling-rules-is-slo...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
Can we extend the Templates in Guvnor
by swaroop
Hi
Inside guvnor when we try to draft a rule we mostly use the default
templates for the LHS and RHS part . Is there any possibility to override
the generated/default templates to augment with some additional
functionality. Like for example for RHS part there would be *"Insert Fact
A"* in the guided editor and when i click that the drl generated in
background inserts Fact A ,i want to do some processing after inserting the
fact , like i want to add the fact to a list .
I evaluated DSL for the above usecase but with DSL i can explicity do some
processing in the background but i not be able to add the functionality of
setting/displaying/adding attributes of the particular fact
Do let me know if you have any suggestions
Regards
Swaroop
--
View this message in context: http://drools.46999.n3.nabble.com/Can-we-extend-the-Templates-in-Guvnor-t...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
Specifying Infinite Limits
by droolster
Hello,
I have the following code to do some interval checking:
declare Classification
low : BigInteger
high : BigInteger
score : int
end
rule "classifications"
when
then
insert (new Classification(new BigInteger("-1000000000000"),
new BigInteger("1000000"), 1));
insert (new Classification(new BigInteger("1000000"), new
BigInteger("5000000"), 2));
insert (new Classification(new BigInteger("5000000"), new
BigInteger("25000000"), 3));
insert (new Classification(new BigInteger("25000000"), new
BigInteger("25000000000000"), 4));
end
rule "Foo"
when
$a: Classification( $low : getLow(), $high : getHigh() )
$b: Data( bar > $low && bar < $high )
then
System.out.println($a.getScore());
end
It works fine but I was wondering if there is a more elegant way to specify
the lowest and highest interval limits. At the moment, as you can see, I
have set it to a very low number and a very high number in the context of my
problem (-1000000000000 and 250000000000000 respectively) . For my problem
specification, these limits should rarely be reached. Can anyone tell me a
robust way of specifying these lowest and highest limits without putting
arbitarily low and high numbers.
Thanks in advance for your help.
Regards.
--
View this message in context: http://drools.46999.n3.nabble.com/Specifying-Infinite-Limits-tp4024627.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
Specifying Infinite Limits
by droolster
Hello,
I have the following code to do some interval checking:
declare Classification
low : BigInteger
high : BigInteger
score : int
end
rule "classifications"
when
then
insert (new Classification(new BigInteger("-1000000000000"), new
BigInteger("1000000"), 1));
insert (new Classification(new BigInteger("1000000"), new
BigInteger("5000000"), 2));
insert (new Classification(new BigInteger("5000000"), new
BigInteger("25000000"), 3));
insert (new Classification(new BigInteger("25000000"), new
BigInteger("25000000000000"), 4));
end
rule "Foo"
when
$a: Classification( $low : getLow(), $high : getHigh() )
$b: Data( bar > $low && bar < $high )
then
System.out.println($a.getScore());
end
It works fine but I was wondering if there is a more elegant way to specify
the lowest and highest interval limits. At the moment, as you can see, I
have set it to a very low number and a very high number in the context of my
problem. For my problem specification, these limits should really be
reached. Can anyone tell me a robust way of specifying these lowest and
highest limits without putting arbitarily low and high numbers.
Thanks in advance for your help.
Regards.
--
View this message in context: http://drools.46999.n3.nabble.com/Specifying-Infinite-Limits-tp4024626.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
Is it possible to update KnowledgeBase inside a rule?
by Sonata
Say I want to change the loaded packages dynamically base on some Business
logic,
is it possible to update the KnowledgeBase in rule files (not in Java) like
calling addKnowledgePackages() removeKnowledgePackage() etc.?
If it is possible, how do you make it so that the current session is updated
with the changes and re-fire for the new rules?
Or, how do you make it so that the current session is not affected but the
next session will be using the updated KnowledgeBase?
Thank you
P.S. I am using 5.5.0.Final
--
View this message in context: http://drools.46999.n3.nabble.com/Is-it-possible-to-update-KnowledgeBase-...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
Insert list of object to Guvnor in Decision table
by Raju Bandaru
Hi,
Am using Drools Guvnor 5.4 final version. I have created a package, decision
table in Guvnor and am invoking decision table from Spring application.
Please find attached snippet for your reference.
KnowledgeBase kbase = readKnowledgeBaseDefault();
StatefulKnowledgeSession ksession = null;
String message = null;
try {
ksession = kbase.newStatefulKnowledgeSession();
*ksession.insert(contactList);*
ksession.fireAllRules();
Requirement for me to send list of object(each objects is a rule :
contactList) to Guvnor and Guvnor should iterate the list of object and
invoke the rule.
Please help me on this requirement and how to achieve?
Regards,
Raju
--
View this message in context: http://drools.46999.n3.nabble.com/Insert-list-of-object-to-Guvnor-in-Deci...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
Getter/Setter Methods for declared types
by droolster
Hello,
At compile time, are the getter and setter methods automatically created for
a declared type. In the Drools Expert PDF it gives the following example:
declare Person
name : String
dateOfBirth : java.util.Date
address : Address
end
and it states that the following class is generated:
public class Person implements Serializable {
private String name;
private java.util.Date dateOfBirth;
private Address address;
// empty constructor
public Person() {...}
// constructor with all fields
public Person( String name, Date dateOfBirth, Address address ) {...}
// if keys are defined, constructor with keys
public Person( ...keys... ) {...}
*// getters and setters*
// equals/hashCode
// toString
}
Then the setter is referenced as follows:
rule "Using a declared Type"
when
$p : Person( name == "Bob" )
then
// Insert Mark, who is Bob's mate.
Person mark = new Person();
*mark.setName("Mark");*
insert( mark );
end
----
I tried the same with this simple declared type:
declare Foo
bar : int
end
rule "abc"
when
some condition
then
Foo e = new Foo();
* e.setBar = 2;*
insert (e);
end
But the compiler complains indicating that it "cannot be resolved or is not
a field".
Do I need to specify the getter/setters in my DRL file explicitly?
Thank you for your help in advance.
Regards
--
View this message in context: http://drools.46999.n3.nabble.com/Getter-Setter-Methods-for-declared-type...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 5 months
