[jboss-jira] [JBoss JIRA] (DROOLS-3535) Out of Memory Error when creating KieSession

[Sarvagya Pant (Jira)]

Sarvagya Pant created DROOLS-3535:
-------------------------------------

             Summary: Out of Memory Error when creating KieSession
                 Key: DROOLS-3535
                 URL: https://issues.jboss.org/browse/DROOLS-3535
             Project: Drools
          Issue Type: Bug
          Components: core engine
    Affects Versions: 7.11.0.Final
         Environment: In MacBook Pro (15-inch, 2017)

JDK 8
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

OpenJDK 11
openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)
            Reporter: Sarvagya Pant
            Assignee: Mario Fusco


We are using Drools compiler to compile DRL and create KieSession. Internally, when we try to compile large number of Spreadsheet (e.g. say 500), each containing about 30 rules, we get Out of memory error as shown in logs below:

java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

	at java.base/java.lang.Thread.start0(Native Method)
	at java.base/java.lang.Thread.start(Thread.java:803)
	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1155)
	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1135)
	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:973)
	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:2554)
	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildPackages(KnowledgeBuilderImpl.java:2489)
	at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:113)
	at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:103)
	at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
	at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
	at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:267)
	at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:235)
	at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:184)
	at org.kie.internal.utils.KieHelper.verify(KieHelper.java:94)
	at abc.xyz.KnowledgeBaseCreator.createKnowledgeBaseFromDrl(KnowledgeBaseCreator.java:319)

This issue can be traced to KnowledgeBuilderImpl.java as shown below:

 Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
            ForkJoinPool pool = new ForkJoinPool(); // avoid common pool
            try {
                pool.submit(() -> 
                rules.stream().parallel()
                        .filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE, ruleDescr.getNamespace(), ruleDescr.getName()))
                        .forEach(ruleDescr -> {
                            initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
                            RuleBuildContext context = buildRuleBuilderContext(pkgRegistry, ruleDescr);
                            ruleCxts.put(ruleDescr.getName(), context);
                            List<? extends KnowledgeBuilderResult> results = addRule(context);
                            if (!results.isEmpty()) {
                            synchronized (this.results) {
                                    this.results.addAll(results);
                                }
                            }
                        })
                ).get();
            } catch (InterruptedException | ExecutionException e) { 
                throw new RuntimeException("Rules compilation failed or interrupted", e);
            }
            for (RuleDescr ruleDescr : rules) {
                RuleBuildContext context = ruleCxts.get(ruleDescr.getName());
                if (context != null) {
                    pkgRegistry.getPackage().addRule(context.getRule());
                }
            }

Here ForkJoinPool is created without specifying any parallelism limit as well as without shutting it down. Moreover, since its avoiding common pool, anyone who is using it in a loop will face the issue as shown below.



--
This message was sent by Atlassian Jira
(v7.12.1#712002)