[jboss-jira] [JBoss JIRA] (DROOLS-3535) Out of Memory Error when creating KieSession
Sarvagya Pant (Jira)
issues at jboss.org
Fri Jan 18 16:33:00 EST 2019
[ https://issues.jboss.org/browse/DROOLS-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sarvagya Pant updated DROOLS-3535:
----------------------------------
Description:
We are using Drools compiler to compile DRL and create KieSession. Internally, when we try to compile large number of Spreadsheet (e.g. say 500), each containing about 30 rules, we get Out of memory error as shown in logs below:
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1155)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1135)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:973)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:2554)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildPackages(KnowledgeBuilderImpl.java:2489)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:113)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:103)
at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:267)
at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:235)
at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:184)
at org.kie.internal.utils.KieHelper.verify(KieHelper.java:94)
at abc.xyz.KnowledgeBaseCreator.createKnowledgeBaseFromDrl(KnowledgeBaseCreator.java:319)
This issue can be traced to KnowledgeBuilderImpl.java as shown below:
Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
ForkJoinPool pool = new ForkJoinPool(); // avoid common pool
try {
pool.submit(() ->
rules.stream().parallel()
.filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE, ruleDescr.getNamespace(), ruleDescr.getName()))
.forEach(ruleDescr -> {
initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
RuleBuildContext context = buildRuleBuilderContext(pkgRegistry, ruleDescr);
ruleCxts.put(ruleDescr.getName(), context);
List<? extends KnowledgeBuilderResult> results = addRule(context);
if (!results.isEmpty()) {
synchronized (this.results) {
this.results.addAll(results);
}
}
})
).get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException("Rules compilation failed or interrupted", e);
}
for (RuleDescr ruleDescr : rules) {
RuleBuildContext context = ruleCxts.get(ruleDescr.getName());
if (context != null) {
pkgRegistry.getPackage().addRule(context.getRule());
}
}
Here ForkJoinPool is created without specifying any parallelism limit as well as without shutting it down. Moreover, since its avoiding common pool, anyone who is using it in a loop will face the issue as discussed above.
was:
We are using Drools compiler to compile DRL and create KieSession. Internally, when we try to compile large number of Spreadsheet (e.g. say 500), each containing about 30 rules, we get Out of memory error as shown in logs below:
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1155)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1135)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:973)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:2554)
at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildPackages(KnowledgeBuilderImpl.java:2489)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:113)
at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:103)
at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:267)
at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:235)
at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:184)
at org.kie.internal.utils.KieHelper.verify(KieHelper.java:94)
at abc.xyz.KnowledgeBaseCreator.createKnowledgeBaseFromDrl(KnowledgeBaseCreator.java:319)
This issue can be traced to KnowledgeBuilderImpl.java as shown below:
Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
ForkJoinPool pool = new ForkJoinPool(); // avoid common pool
try {
pool.submit(() ->
rules.stream().parallel()
.filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE, ruleDescr.getNamespace(), ruleDescr.getName()))
.forEach(ruleDescr -> {
initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
RuleBuildContext context = buildRuleBuilderContext(pkgRegistry, ruleDescr);
ruleCxts.put(ruleDescr.getName(), context);
List<? extends KnowledgeBuilderResult> results = addRule(context);
if (!results.isEmpty()) {
synchronized (this.results) {
this.results.addAll(results);
}
}
})
).get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException("Rules compilation failed or interrupted", e);
}
for (RuleDescr ruleDescr : rules) {
RuleBuildContext context = ruleCxts.get(ruleDescr.getName());
if (context != null) {
pkgRegistry.getPackage().addRule(context.getRule());
}
}
Here ForkJoinPool is created without specifying any parallelism limit as well as without shutting it down. Moreover, since its avoiding common pool, anyone who is using it in a loop will face the issue as shown below.
> Out of Memory Error when creating KieSession
> --------------------------------------------
>
> Key: DROOLS-3535
> URL: https://issues.jboss.org/browse/DROOLS-3535
> Project: Drools
> Issue Type: Bug
> Components: core engine
> Affects Versions: 7.11.0.Final
> Environment: In MacBook Pro (15-inch, 2017)
> JDK 8
> java version "1.8.0_161"
> Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
> OpenJDK 11
> openjdk version "11.0.1" 2018-10-16
> OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
> OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)
> Reporter: Sarvagya Pant
> Assignee: Mario Fusco
> Priority: Blocker
>
> We are using Drools compiler to compile DRL and create KieSession. Internally, when we try to compile large number of Spreadsheet (e.g. say 500), each containing about 30 rules, we get Out of memory error as shown in logs below:
> java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
> at java.base/java.lang.Thread.start0(Native Method)
> at java.base/java.lang.Thread.start(Thread.java:803)
> at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1155)
> at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1135)
> at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:973)
> at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:2554)
> at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildPackages(KnowledgeBuilderImpl.java:2489)
> at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:113)
> at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:103)
> at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
> at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
> at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:267)
> at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:235)
> at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:184)
> at org.kie.internal.utils.KieHelper.verify(KieHelper.java:94)
> at abc.xyz.KnowledgeBaseCreator.createKnowledgeBaseFromDrl(KnowledgeBaseCreator.java:319)
> This issue can be traced to KnowledgeBuilderImpl.java as shown below:
> Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
> ForkJoinPool pool = new ForkJoinPool(); // avoid common pool
> try {
> pool.submit(() ->
> rules.stream().parallel()
> .filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE, ruleDescr.getNamespace(), ruleDescr.getName()))
> .forEach(ruleDescr -> {
> initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
> RuleBuildContext context = buildRuleBuilderContext(pkgRegistry, ruleDescr);
> ruleCxts.put(ruleDescr.getName(), context);
> List<? extends KnowledgeBuilderResult> results = addRule(context);
> if (!results.isEmpty()) {
> synchronized (this.results) {
> this.results.addAll(results);
> }
> }
> })
> ).get();
> } catch (InterruptedException | ExecutionException e) {
> throw new RuntimeException("Rules compilation failed or interrupted", e);
> }
> for (RuleDescr ruleDescr : rules) {
> RuleBuildContext context = ruleCxts.get(ruleDescr.getName());
> if (context != null) {
> pkgRegistry.getPackage().addRule(context.getRule());
> }
> }
> Here ForkJoinPool is created without specifying any parallelism limit as well as without shutting it down. Moreover, since its avoiding common pool, anyone who is using it in a loop will face the issue as discussed above.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
More information about the jboss-jira
mailing list