2:38 a.m.
Has anyone out there experienced any slowdowns using Drools 5.2 to build a
knowledge base? I have around 700 drl files (containing anywhere from 1 to
30 rules each) and the time to build a KnowledgeBase with them has increased
from about 10 minutes in Drools 5.1 to 90 minutes in Drools 5.2. I realize I
have a slow computer here, but the time difference between the two versions
is staggering.
I have already been serializing the KnowledgeBase object for performance
reasons and even loading the serialized KnowledgeBase takes about 3-4x
longer in 5.2 than in 5.1. Am I doing something wrong here? Here are some
code snippets of how I am building and loading the kbase:
//////////////////////// Building
final KnowledgeBuilder kBuilder =
KnowledgeBuilderFactory.newKnowledgeBuilder();
final List<File> drlFiles = getAllDrlFiles(directory);
for (final File name : drlFiles) {
kBuilder.add(ResourceFactory.newFileResource(name),
ResourceType.DRL);
}
if (kBuilder.hasErrors()) {
// ....
}
final KnowledgeBase kBase = KnowledgeBaseFactory.newKnowledgeBase();
kBase.addKnowledgePackages(kBuilder.getKnowledgePackages());
ObjectOutputStream oos = null;
try {
oos = new DroolsObjectOutputStream(new FileOutputStream(pkgFile));
oos.writeObject(kBase);
} catch (...) {
// ....
}
//////////////////////// Loading
ois = new DroolsObjectInputStream(new FileInputStream(pkgFile));
final KnowledgeBase kBase = (KnowledgeBase) ois.readObject();
final StatefulKnowledgeSession ksession =
kBase.newStatefulKnowledgeSession();
Thanks!
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
2:57 a.m.
We believe this has been introduced to MVEL hitting the classloader too
much. Mario is looking into fixing this.
Mark
On 09/08/2011 18:38, darktyco wrote:
Has anyone out there experienced any slowdowns using Drools 5.2 to
build a
knowledge base? I have around 700 drl files (containing anywhere from 1 to
30 rules each) and the time to build a KnowledgeBase with them has increased
from about 10 minutes in Drools 5.1 to 90 minutes in Drools 5.2. I realize I
have a slow computer here, but the time difference between the two versions
is staggering.
I have already been serializing the KnowledgeBase object for performance
reasons and even loading the serialized KnowledgeBase takes about 3-4x
longer in 5.2 than in 5.1. Am I doing something wrong here? Here are some
code snippets of how I am building and loading the kbase:
//////////////////////// Building
final KnowledgeBuilder kBuilder =
KnowledgeBuilderFactory.newKnowledgeBuilder();
final List<File> drlFiles = getAllDrlFiles(directory);
for (final File name : drlFiles) {
kBuilder.add(ResourceFactory.newFileResource(name),
ResourceType.DRL);
}
if (kBuilder.hasErrors()) {
// ....
}
final KnowledgeBase kBase = KnowledgeBaseFactory.newKnowledgeBase();
kBase.addKnowledgePackages(kBuilder.getKnowledgePackages());
ObjectOutputStream oos = null;
try {
oos = new DroolsObjectOutputStream(new FileOutputStream(pkgFile));
oos.writeObject(kBase);
} catch (...) {
// ....
}
//////////////////////// Loading
ois = new DroolsObjectInputStream(new FileInputStream(pkgFile));
final KnowledgeBase kBase = (KnowledgeBase) ois.readObject();
final StatefulKnowledgeSession ksession =
kBase.newStatefulKnowledgeSession();
Thanks!
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
1:02 a.m.
I did some benchmarking yesterday and noticed this too. I also found that
package load time is much slower when using change sets.
originally my drools-spring file looked like this (snippet):
.....
<drools:kagent id="venueKnowledgeAgent" kbase="venueKnowledgeBase"
new-instance="false">
<drools:resources>
<drools:resource type="CHANGE_SET"
source="file:///usr/local/tomcat-rules/webapps/rules/WEB-INF
/VenueChangeSet.xml"/>
</drools:resources>
</drools:kagent>
.....
and the change set was:
<?xml version="1.0" encoding="UTF-8" ?>
<change-set xmlns="http://drools.org/drools-5.0/change-set"
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xs:schemaLocation="http://drools.org/drools-5.0/change-set
http://anonsvn.jboss.org/repos/labs/labs/jbossrules/trunk/drools-api/src/...
<add>
<resource type="PKG"
source="file:///usr/local/tomcat-rules/webapps/rules/packages/venue" />
</add>
</change-set>
when I got rid of the change set, and changed the drools-spring to this:
<drools:kagent id="venueKnowledgeAgent" kbase="venueKnowledgeBase"
new-instance="false">
<drools:resources>
<drools:resource type="PKG"
source="file:///usr/local/tomcat-rules/webapps/rules/packages/venue"/>
</drools:resources>
</drools:kagent>
the packages loaded in half the time.
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12:39 a.m.
Great, thanks for the heads up. I didn't see anything in your JIRA for it,
would it help if I created an issue? If needed I am available to help test
as well.
Thanks
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12:16 p.m.
In case anyone else is struggling with this problem I found that using the
latest beta mvel jar (beta 6) helps a lot. Performance and memory use
building and loading my large KnowledgeBase is much better- on my slower
machine I'm down to 20 minutes (from 60) of compile time. Not back to Drools
5.1 (10 minutes) speed. and I still have to give JBoss extra memory to
deploy my webapp, but still much better.
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
6:41 p.m.
Cheers, thanks for the info, that does greatly improve compilation time - I take it it is
your maxpermsize that you had to increase in jboss?
Thomas
-----Original Message-----
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-
bounces(a)lists.jboss.org] On Behalf Of darktyco
Sent: 24 August 2011 04:17
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Drools 5.2 performance building KnowledgeBase
In case anyone else is struggling with this problem I found that using the
latest beta mvel jar (beta 6) helps a lot. Performance and memory use
building and loading my large KnowledgeBase is much better- on my slower
machine I'm down to 20 minutes (from 60) of compile time. Not back to
Drools
5.1 (10 minutes) speed. and I still have to give JBoss extra memory to deploy
my webapp, but still much better.
--
View this message in context: http://drools.46999.n3.nabble.com/Drools-5-
2-performance-building-KnowledgeBase-tp3239746p3279902.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the postmaster(a)nds.com and delete it from
your system as well as any copies. The content of e-mails as well as traffic data may be
monitored by NDS for employment and security purposes. To protect the environment please
do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
3:25 a.m.
I don't see a Beta6 jar. The one in Drools 5.3 is mvel2-2.1.Beta5.jar -- is
that the one you mean? Are you at drools 5.3 beta1 code level?
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
10:58 a.m.
I got it directly from the MVEL Web site:
http://mvel.codehaus.org/Downloading+MVEL
http://mvel.codehaus.org/Downloading+MVEL
Good luck! Hopefully this version, or an even better one, will be included
with Drools 5.3 final.
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
11:14 a.m.
I forgot to answer your other question: I'm on Drools 5.2. I had tried the
Drools 5.3 Beta 1 distribution but there was no improvement- it seems
something was fixed specifically in MVEL 2.1 Beta6.
Also, to Thomas: I have to specify -Xmx512m in order to run a pretty bare
minimum unit test (this does include a Spring application context but it is
very small) using this Drools setup to run or I will get an OOM Java Heap
exception within seconds. It is actually kind of strange how quickly the
exception is thrown. I will double check the 512 figure and get back to you
if I can indeed get it to run with less memory, I really dislike spreading
bad info around. The full Webapp needs more than that plus a MaxPermSize
increase, but there is a lot of other stuff going on in the war.
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
5:08 p.m.
512mb isn't that big a number, I found that it's the MaxPermSize that was
actually
more important to increase with the compilation as a lot of time the usage was
bouncing just underneath the default limit on my system (80mb).
Giving it more headroom reduced the amount of effort the garbage collector was
doing to stop the limit actually being reached.
If you are using maven just add an explicit dependency on mvel in the
dependencyManagement section of your pom:
<dependency>
<groupId>org.mvel</groupId>
<artifactId>mvel2</artifactId>
<version>2.1.Beta6</version>
</dependency>
If your using surefire to run your unit tests with maven you may want to add the
appropriate arguments to give the forked jvm more memory, this can be done by
defining the argLine property
<properties>
<!--Make surefire fork with at least 256mb of heap and 128mb perm gen
space-->
<argLine>-Xmx256m -XX:MaxPermSize=128m</argLine>
</properties>
Thomas
-----Original Message-----
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-
bounces(a)lists.jboss.org] On Behalf Of darktyco
Sent: 25 August 2011 03:15
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Drools 5.2 performance building KnowledgeBase
I forgot to answer your other question: I'm on Drools 5.2. I had tried the
Drools 5.3 Beta 1 distribution but there was no improvement- it seems
something was fixed specifically in MVEL 2.1 Beta6.
Also, to Thomas: I have to specify -Xmx512m in order to run a pretty bare
minimum unit test (this does include a Spring application context but it is very
small) using this Drools setup to run or I will get an OOM Java Heap exception
within seconds. It is actually kind of strange how quickly the exception is
thrown. I will double check the 512 figure and get back to you if I can indeed
get it to run with less memory, I really dislike spreading bad info around. The
full Webapp needs more than that plus a MaxPermSize increase, but there is
a lot of other stuff going on in the war.
--
View this message in context: http://drools.46999.n3.nabble.com/Drools-5-
2-performance-building-KnowledgeBase-tp3239746p3282810.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the postmaster(a)nds.com and delete it from
your system as well as any copies. The content of e-mails as well as traffic data may be
monitored by NDS for employment and security purposes. To protect the environment please
do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
1:10 a.m.
Thanks - I got the mvel2-2.1.Beta6.jar and added it to my 5.2.0 Drools lib
instead of the drools mvel jar. Compile time on large rules spreadsheets
went down by 50%, and load time into knowledge base of those spreadsheet
packages went down by about 75%. Everything else seems to work fine with
the beta6 jar. Thanks for your help! I hope they include this in 5.3.
--
View this message in context:
http://drools.46999.n3.nabble.com/Drools-5-2-performance-building-Knowled...
Sent from the Drools: User forum mailing list archive at Nabble.com.
4639
days inactive
4655
days old
10 comments
4 participants
participants (4)
-
darktyco -
lhorton -
Mark Proctor -
Swindells, Thomas