]
Mario Fusco updated DROOLS-3535:
--------------------------------
Sprint: 2019 Week 05-07
Out of Memory Error when creating KieSession
--------------------------------------------
Key: DROOLS-3535
URL:
https://issues.jboss.org/browse/DROOLS-3535
Project: Drools
Issue Type: Bug
Components: core engine
Affects Versions: 7.11.0.Final
Environment: In MacBook Pro (15-inch, 2017)
JDK 8
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
OpenJDK 11
openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)
Reporter: Sarvagya Pant
Assignee: Mario Fusco
Priority: Blocker
We are using Drools compiler to compile DRL and create KieSession. Internally, when we
try to compile large number of Spreadsheet (e.g. say 500), each containing about 30 rules,
we get Out of memory error as shown in logs below:
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or
process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at
org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1155)
at
org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1135)
at
org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:973)
at
org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:2554)
at
org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildPackages(KnowledgeBuilderImpl.java:2489)
at
org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:113)
at
org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:103)
at
org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
at
org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
at
org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:267)
at
org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:235)
at
org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:184)
at org.kie.internal.utils.KieHelper.verify(KieHelper.java:94)
at
abc.xyz.KnowledgeBaseCreator.createKnowledgeBaseFromDrl(KnowledgeBaseCreator.java:319)
This issue can be traced to KnowledgeBuilderImpl.java as shown below:
Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
ForkJoinPool pool = new ForkJoinPool(); // avoid common pool
try {
pool.submit(() ->
rules.stream().parallel()
.filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE,
ruleDescr.getNamespace(), ruleDescr.getName()))
.forEach(ruleDescr -> {
initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
RuleBuildContext context =
buildRuleBuilderContext(pkgRegistry, ruleDescr);
ruleCxts.put(ruleDescr.getName(), context);
List<? extends KnowledgeBuilderResult> results =
addRule(context);
if (!results.isEmpty()) {
synchronized (this.results) {
this.results.addAll(results);
}
}
})
).get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException("Rules compilation failed or
interrupted", e);
}
for (RuleDescr ruleDescr : rules) {
RuleBuildContext context = ruleCxts.get(ruleDescr.getName());
if (context != null) {
pkgRegistry.getPackage().addRule(context.getRule());
}
}
Here ForkJoinPool is created without specifying any parallelism limit as well as without
shutting it down. Moreover, since its avoiding common pool, anyone who is using it in a
loop will face the issue as discussed above.