8:31 a.m.
Hello,
I am trying to create a bunch of KnowledgeBase objects, this is what I am
doing:
1. create one List of KnowledgePackage objects (pre-compile them).
2. Create X number of KnowledgeBases, and for each knowledge base do this:
for (KnowledgePackage kp : this.knowledgePackages) {
Collection<KnowledgePackage> collections = new
ArrayList<KnowledgePackage>();
collections.add(kp);
knowledgeBase.addKnowledgePackages(collections);
}
Please forget about optimizations for now, this is just an isolated test to
solve this issue. So, notice how I am using the SAME list of
KnowledgePackage objects, but to initialize MULTIPLE KnowledgeBase
instances. Here is the exception that I am getting (and sometimes I get it
sometimes I don't on the same code!). This exception happens exactly at the
addKnowledgePackage line (last line in the loop above). One last piece of
information, I am executing the code above in a thread, meaning multiple
threads are accessing the same KnowledgePackage array list but populating
totally distinct KnowledgeBases. Any idea any one?
Exception in thread "KnowledgeBasePoolObject_Init_1"
java.lang.ArrayIndexOutOfBo
undsException: 7
at java.util.ArrayList.add(Unknown Source)
at org.drools.rule.EvalCondition.clone(EvalCondition.java:135)
at
org.drools.rule.LogicTransformer$AndOrTransformation.transform(LogicT
ransformer.java:330)
at
org.drools.rule.LogicTransformer.applyOrTransformation(LogicTransform
er.java:242)
at
org.drools.rule.LogicTransformer.processTree(LogicTransformer.java:23
2)
at
org.drools.rule.LogicTransformer.transform(LogicTransformer.java:81)
at org.drools.rule.Rule.getTransformedLhs(Rule.java:523)
at
org.drools.reteoo.builder.ReteooRuleBuilder.addRule(ReteooRuleBuilder
.java:97)
at org.drools.reteoo.ReteooBuilder.addRule(ReteooBuilder.java:117)
at org.drools.reteoo.ReteooRuleBase.addRule(ReteooRuleBase.java:409)
at
org.drools.common.AbstractRuleBase.addRule(AbstractRuleBase.java:635)
at
org.drools.common.AbstractRuleBase.addPackages(AbstractRuleBase.java:
517)
at
org.drools.reteoo.ReteooRuleBase.addPackages(ReteooRuleBase.java:426)
at
org.drools.impl.KnowledgeBaseImpl.addKnowledgePackages(KnowledgeBaseI
mpl.java:121)
at
com.medcpu.mpu.pp.engine.KnowledgeBasePool.getNewKnowledgeBasePoolObj
ect(KnowledgeBasePool.java:125)
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/ArrayIndexOutOfBounds...
Sent from the Drools - User mailing list archive at Nabble.com.
8:39 a.m.
And for the same isolated test, but in a different run, I also get this
(exactly at the same line):
P.S. I was just trying to create 7 KnowledgeBase instances. I am also
running with a huge perm memory setting: -XX:PermSize=128m
-XX:MaxPermSize=672
Exception in thread "main" java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(Unknown Source)
at
org.drools.rule.JavaDialectRuntimeData$PackageClassLoader.fastFindCla
ss(JavaDialectRuntimeData.java:440)
at
org.drools.rule.CompositeClassLoader.fastFindClass(CompositeClassLoad
er.java:53)
at
org.drools.rule.CompositeClassLoader.loadClass(CompositeClassLoader.j
ava:68)
at java.lang.ClassLoader.loadClass(Unknown Source)
at
org.drools.rule.JavaDialectRuntimeData.wire(JavaDialectRuntimeData.ja
va:306)
at
org.drools.rule.JavaDialectRuntimeData.wire(JavaDialectRuntimeData.ja
va:297)
at
org.drools.rule.JavaDialectRuntimeData.onBeforeExecute(JavaDialectRun
timeData.java:143)
at
org.drools.rule.DialectRuntimeRegistry.onBeforeExecute(DialectRuntime
Registry.java:114)
at
org.drools.common.AbstractRuleBase.addPackages(AbstractRuleBase.java:
481)
at
org.drools.reteoo.ReteooRuleBase.addPackages(ReteooRuleBase.java:426)
at
org.drools.impl.KnowledgeBaseImpl.addKnowledgePackages(KnowledgeBaseI
mpl.java:121)
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/ArrayIndexOutOfBounds...
Sent from the Drools - User mailing list archive at Nabble.com.
9:28 a.m.
Additional information. When I removed the threads (everything executing
sequentially), I got rid of the first problem above which is the index out
of bounds exception. However, I am consistently getting the perm gen out of
memory exception although I have plenty of it.
So, I decided to create those 7 instances of KnowledgeBase objects and not
save them in memory (leave them eligible for GC) and I was able to get up to
20 instances created. When I added the statement that adds every newly
created KnowledgeBase to an existing array list, that is when I got the
memory exception again at around 5/6 KnowledgeBase.
If I try to create KnowledgeBase instances WITHOUT adding KnowledgePackages
to it (I am adding a total of 6), then I can create all KnowledgeBase
successfully.
I even created a whole new KnolwedgePackage array list from scratch on every
creation of KnowledgeBase just to isolate any potential problems due to
sharing of the array list among the newly created KnowledgeBases, and the
issue remains the same (memory error).
So, it is not the sharing of the array list of KnowledgePackage instances,
it is not the fact that I am creating multiple KnowledgeBases in memory. It
is the fact that I am creating multiple instances of KnowledgeBase objects
AND adding 6 KnowledgePackage objects to each one. I max out at around 5/6
and then it fails.
Here is the last run that failed (I printed out Runtime...freeMemory() and
availableMemory()):
Free memory: 45 MB
Available memory: 254 MB
Then the exception right when I wanted to add KP array list to the next
KnowledgeBase.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/ArrayIndexOutOfBounds...
Sent from the Drools - User mailing list archive at Nabble.com.
8:33 p.m.
For those that are interested. I solved the problem. Basically I got over the
following inherent problems in Drools (not that they are bugs, but they did
not suit my application requirements):
1. A new class loader for each KnowledgeBase. This caused about 20-30K
classes loaded per KnowledgeBase causing me the Perm gen errors. If I
increased the Perm gen memory parameters, I got hit on the native thread
creation issue (the more memory your application consumes, the less threads
you can instantiate because you will have less memory to work with).
2. The time taken to compile the KnowledgePackages AND to add them to each
KnowledgeBase.
So, what I did was a combination of a few things:
1. I created one KnowledgeBase and loaded all the KnowledgePackages that I
needed to load in it. Serialized it to a file. And retrieved X instances and
added them to a pool. Now, I have X KnowledgeBase instances without the
heavy class loading that comes with creating each one if you use the regular
way of creating it from scratch. When you de-serialize it, there are a lot
less classes loaded. Doing this also allowed me to avoid compiling the
KnowledgePackages on start up, and now my application starts up immediately
like a bullet :)
2. I changed some of my JVM parameters to add very aggressive GC parameters.
The fact that I was able to avoid using many class loaders for each
instantiation of a KnowledgeBase allowed me to reduce the Perm Gen memory
setting which allowed me more memory for my threads :) my application is
actually thread heavy, and why not when we have an 8-core server!!
I am using the following JVM parameters for anyone interested:
-XX:+CMSClassUnloadingEnabled -XX:+UseParallelOldGC -XX:ParallelGCThreads=8
You can look up their definitions. Now I am golden!!!
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/ArrayIndexOutOfBounds...
Sent from the Drools - User mailing list archive at Nabble.com.
12:55 a.m.
On 23 May 2010 15:31, malkhafaji <moe.alkhafaji(a)medcpu.com> wrote:
Hello,
I am trying to create a bunch of KnowledgeBase objects, this is what I am
doing:
1. create one List of KnowledgePackage objects (pre-compile them).
2. Create X number of KnowledgeBases, and for each knowledge base do this:
for (KnowledgePackage kp : this.knowledgePackages) {
Collection<KnowledgePackage> collections =
new
ArrayList<KnowledgePackage>();
collections.add(kp);
knowledgeBase.addKnowledgePackages(collections);
}
This is a very circumstantial way of doing
knowledgeBase.addKnowledgePackages( this.knowledgePackages );
-W
5:08 a.m.
Yes. Actually that is exactly what I did, but I wanted to test something out
in the process and I decided to try it like that. But in any case, I am
using your line instead. I wrote the disclaimer in the email I sent with the
code :)
On Mon, May 24, 2010 at 12:59 AM, Wolfgang Laun-2 [via Drools - Java Rules
Engine]
<ml-node+838997-848786506-94606@n3.nabble.com<ml-node%2B838997-848786506-94606(a)n3.nabble.com>
wrote:
> On 23 May 2010 15:31, malkhafaji <[hidden
email]<http://user/SendEmail.jtp?type=node&node=838997&i=0>
> wrote:
>
>>
>> Hello,
>>
>> I am trying to create a bunch of KnowledgeBase objects, this is what I am
>> doing:
>>
>> 1. create one List of KnowledgePackage objects (pre-compile them).
>> 2. Create X number of KnowledgeBases, and for each knowledge base do this:
>>
>> for (KnowledgePackage kp : this.knowledgePackages) {
>> Collection<KnowledgePackage> collections =
>> new
>> ArrayList<KnowledgePackage>();
>> collections.add(kp);
>>
>> knowledgeBase.addKnowledgePackages(collections);
>> }
>>
>
> This is a very circumstantial way of doing
>
> knowledgeBase.addKnowledgePackages( this.knowledgePackages );
>
> -W
>
> _______________________________________________
> rules-users mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=838997&i=1>
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
> ------------------------------
> View message @
>
http://drools-java-rules-engine.46999.n3.nabble.com/ArrayIndexOutOfBounds...
> To unsubscribe from ArrayIndexOutOfBounds when calling
> addKnowledgePackage?, click
here<http://drools-java-rules-engine.46999.n3.nabble.com/subscriptions...;.
>
>
>
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/ArrayIndexOutOfBounds...
Sent from the Drools - User mailing list archive at Nabble.com.
5816
days inactive
5817
days old
5 comments
2 participants
participants (2)
-
malkhafaji -
Wolfgang Laun