Possible memory leak in Drools 6.0.1
by kellyajp
We have upgraded from Drools 5 -> Drools 6.01 and when repeating the
processing in each environment we have very different object counts that is
causing memory to run out on Drools 6:
Below is an extract from a heap dump
In Drools 6 the main objects in the heap are
1: 4378006 700480960 org.drools.core.reteoo.RightTuple
Where as in Drools 5 there are way less objects
92: 19022 1826112 org.drools.reteoo.RightTuple
Has anyone else had any experience of this issue?
Thanks
--
View this message in context: http://drools.46999.n3.nabble.com/Possible-memory-leak-in-Drools-6-0-1-tp...
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
10 years, 9 months
Fwd: [rules-users] Drools Fusion inconsistencies at increasing event throughputs
by Wolfgang Laun
Once more, a nagging. How much time could be saved if the docs would
tell the FULL truth, here w.r.t. timers in rules.
I can't even fix it, because this can't be derived easily from the
code. And, maybe, the code doesn't reflect the intent?!
-W
---------- Forwarded message ----------
From: Vieri <vieri.emiliani(a)gmail.com>
Date: Tue, 25 Mar 2014 08:28:43 +0100
Subject: [rules-users] Drools Fusion inconsistencies at increasing
event throughputs
To: rules-users(a)lists.jboss.org
Dear Drools Experts,
*Short version*
1. Cron-based rules triggers (more than once) for a full second, rather
than once every defined period;
2. After some time, event creation from the drl seems to hang. The "time
to hang" decreases as we increase the throughput of incoming events;
3. The two issues seem to be somehow related (well, maybe).
*Full version*
We are testing Drools Fusion to implement CEP functionalities in our
platform. We are performing these tests using Drools 6.0.1.Final.
As a basic test case, we set up a synthetic stream of events, and defined a
couple of rules to implement a simple throughput metric. Here's the
complete drl:
*package* it.intext.unity.test
*import* it.intext.unity.test.SynthEvent;
*import* java.util.Date;
*global* org.slf4j.Logger logger;
*declare* SynthEvent
@role( event )
@timestamp( timestamp )
*end*
*declare* EventCounter
@role( event )
@timestamp( timestamp )
id : *long*
key : String
timestamp : Date
*end*
// Business rules
*rule* "Create counter"
*when*
$e : SynthEvent() *from* entry-point "synth"
*then*
entryPoints["counters"].*insert*(*new* EventCounter( $e.getId(), "event",
$e.getTimestamp() ) );
*end*
// Metrics
*rule* "Count epm"
// Emit count every 10s, accumulate over 1m
timer ( cron: 0/10 * * * * ? )
*when*
Number( $count : intValue ) *from* *accumulate*(
EventCounter( key == "event" ) over window:time( 60s
)*from*entry-point
"counters", count(1) )
*then*
logger.debug("epm = {}", $count );
*end*
The SynthEvent class is very basic:
*public* *class* SynthEvent {
*long* id;
Date timestamp;
List<String> meta;
... // Getters/Setters and constructors omitted
}
Now, the test is performed running the session in one thread (code below)
*private* *void* process(*final* KieSession session) {
*new* Thread(){
*public* *void* run() {
session.fireUntilHalt();
};
}.start();
}
while feeding the events on a second thread (code below)
*private* *void* feed(*final* KieSession session) {
*new* Thread(){
@SuppressWarnings("unchecked")
*public* *void* run() {
*try* {
*int* counter = 0;
*while*(*true*) {
counter++;
session.execute(CommandFactory.
*newInsert*(createEvent(), *null*, *false*, "synth"));
Thread.*sleep*(getSleepRate());
*if* ((counter % 1000) == 0) {
*logger*.debug("Total events: {}",
counter);
}
}
} *catch* (InterruptedException e) {
*logger*.warn("{}", e);
}
};
}.start();
}
I expected the cron rule "count epm" to trigger once every 10 seconds. But
here's an extract from the log (running at 5 events per second +/- 20%):
[DEBUG] 2014-03-25 04:56:10.008
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 1
[DEBUG] 2014-03-25 04:56:10.075
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2
[DEBUG] 2014-03-25 04:56:10.262
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 3
[DEBUG] 2014-03-25 04:56:10.507
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 4
[DEBUG] 2014-03-25 04:56:10.678
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 5
[DEBUG] 2014-03-25 04:56:10.871
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 6
[DEBUG] 2014-03-25 04:56:20.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 50
[DEBUG] 2014-03-25 04:56:20.042
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 51
[DEBUG] 2014-03-25 04:56:20.231
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 52
[DEBUG] 2014-03-25 04:56:20.405
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 53
[DEBUG] 2014-03-25 04:56:20.647
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 54
[DEBUG] 2014-03-25 04:56:20.823
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 55
[DEBUG] 2014-03-25 04:56:20.992
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 56
As you can see, the "count epm" rules fires once per incoming event for a
full second (e.g. for the second cycle we see the first activation at
04:56:20.001 and the last one at 04:56:20.992).
This is not a major issue, since it can be solved with a sort-of
publish/subscribe pattern which, BTW, increases the system stability (if
anyone is interested I can post more details on the list).
Anyhow, the test ran for more than 20' consistently until we decided it was
enough.
Now, the problem is that as we increase the throughput (e.g. 50 events per
second), the test runs for a few minutes before it becomes inconsistent.
The evidence is that the first rule stops to trigger, and EventCounter
aren't created anymore. Here's a log extract:
[DEBUG] 2014-03-25 07:37:40.008
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 199
[DEBUG] 2014-03-25 07:37:40.017
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 200
[DEBUG] 2014-03-25 07:37:40.034
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 201
[DEBUG] 2014-03-25 07:37:40.058
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 202
[DEBUG] 2014-03-25 07:37:40.076
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 203
*... here the system is stable as we get about 3K events per minute as
expected*
[DEBUG] 2014-03-25 07:38:36.022 (StreamTester.java:run:70) Total events:
3000
[DEBUG] 2014-03-25 07:38:40.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2999
[DEBUG] 2014-03-25 07:38:40.016
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2998
[DEBUG] 2014-03-25 07:38:40.024
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2999
[DEBUG] 2014-03-25 07:38:40.034
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2998
[DEBUG] 2014-03-25 07:38:40.047
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2999
[DEBUG] 2014-03-25 07:38:40.057
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2998
...
[DEBUG] 2014-03-25 07:39:50.974
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 3008
[DEBUG] 2014-03-25 07:39:50.988
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 3007
[DEBUG] 2014-03-25 07:39:50.996
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 3008
[DEBUG] 2014-03-25 07:39:55.842 (StreamTester.java:run:70) Total events:
7000
*... Somewhere in between the system becomes inconsistent and the number of
events decreases to 0*
[DEBUG] 2014-03-25 07:40:00.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2734
[DEBUG] 2014-03-25 07:40:00.017
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2733
[DEBUG] 2014-03-25 07:40:00.037
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 2732
...
[DEBUG] 2014-03-25 07:40:50.937
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 181
[DEBUG] 2014-03-25 07:40:50.953
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 180
[DEBUG] 2014-03-25 07:40:50.975
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 179
[DEBUG] 2014-03-25 07:40:50.997
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 178
[DEBUG] 2014-03-25 07:40:55.438 (StreamTester.java:run:70) Total events:
10000
[DEBUG] 2014-03-25 07:41:00.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 0
[DEBUG] 2014-03-25 07:41:10.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 0
We added a quick rule to count "live EventCounter"
*rule* "Count live counters"
timer ( cron: 0/60 * * * * ? )
*when*
Number( $count : intValue ) *from* *accumulate*(
EventCounter( key == "event" ) *from* entry-point "counters",
count(1) )
*then*
logger.debug("Live counters = {}", $count );
*end*
and, again, the evidence is that when things go wrong the EventCounter
events aren't created anymore.
[DEBUG] 2014-03-25 07:40:55.438 (StreamTester.java:run:70) Total events:
10000
[DEBUG] 2014-03-25 07:41:00.001
(Rule_Count_live_counters1625367465.java:defaultConsequence:14) Live
counters = 0
[DEBUG] 2014-03-25 07:41:00.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 0
[DEBUG] 2014-03-25 07:41:10.001
(Rule_Count_epm442808096.java:defaultConsequence:14) epm = 0
Adding a debug line in the "Create counter" rule confirm this hypotesis
(after a while the rule won't trigger anymore). Modified rule is:
*rule* "Create counter"
*when*
$e : SynthEvent() *from* entry-point "synth"
*then*
entryPoints["counters"].*insert*( *new* EventCounter( $e.getId(),
"event", $e.getTimestamp() ) );
*if* (Math.random() < 0.01) logger.debug("New event: {}", $e.getId());
*end*
When the system becomes inconsistent, we stop seeing "New event" log lines.
What are we missing? Any suggestion? Anyone experienced the same problems
we have?
Your help will be greatly appreciated.
Thanks in advance,
Vieri
10 years, 9 months
Pull request for remote deploy (GUVNOR-2088)
by Joe White
I have created a pull request that updates the GuvnorM2Repository code to deploy to a remote repository if one is configured in the <distributionManagement> section of a pom in the workbench. If the repository requires authentication the system will fetch the credentials by repository id from the settings.xml file.
I tested this publishing to both a snapshot and a release repository. Both require credentials and both worked (thanks to mfusco).
One question:
Currently it does not publish to the default profile repositories in the settings.xml. Should it? In other words, if the user has a default profile in their settings should Guvnor go ahead and deploy to the configured repository? This would fit with normal maven operation but I don't know if the user would expect the workbench code to be handled differently.
If you guys want Guvnor to publish to the settings.xml repositories automatically let me know and I can make that update as well.
Once the above is decided I'll update the documentation and create a pull request for that as well.
https://github.com/droolsjbpm/guvnor/pull/111
Joe
10 years, 9 months
Re: [rules-dev] part time intern @ Red Hat
by zeeshan.spring＠gmail.com
I am interested Mark.
Sent from my HTC
----- Reply message -----
From: "Mark Proctor" <mproctor(a)codehaus.org>
To: "Rules Users List" <rules-users(a)lists.jboss.org>, "Rules Dev List" <rules-dev(a)lists.jboss.org>
Subject: [rules-dev] part time intern @ Red Hat
Date: Thu, Mar 20, 2014 9:28 PM
I have a small budget for a part time intern for one year, 12K USD. The work can be 100% remote, from almost any country. This will mostly be UI work, improving our workbench.
If this interests you, email me off list.
Mark
_______________________________________________
rules-dev mailing list
rules-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-dev
10 years, 9 months
Re: [rules-dev] [rules-users] Updating project repositories
by Joe White
Moving to dev list.
Mike,
I spent some time reviewing GuvnorM2Repository with Mark's help. It looks like the workbench will never deploy to a remote repository and it can't currently be configured to do so because of how it gets the deploy URL. Right now it mixes a local folder with the URL so there isn't any way to change that to an external location. It will only deploy to the http repository exposed by Guvnor. Details are in the JIRA.
I believe that the tool should inspect the pom for the package and see if there are repositories and deploy to those. Or it should use the repositories configured in settings.xml for the maven install. Or both.
I opened GUVNOR-2088 for this. Ping me on IRC tomorrow and I may take a shot at fixing this and issuing a pull.
Joe
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On Behalf Of Michael Anstis
Sent: Thursday, March 20, 2014 4:03 PM
To: Rules Users List
Subject: Re: [rules-users] Updating project repositories
The workbench attempts an install and deploy. GuvnorM2Repository in the guvnor project is the class (IIRC).
Sent on the move
On 20 Mar 2014 21:47, "Joe White" <Joe.White(a)recondotech.com<mailto:Joe.White@recondotech.com>> wrote:
In fact, if I manually go into the guvnor repository, copy the pom.xml directly from the workbench, and command line run "mvn deploy" the jar gets pushed out to my remote repository. So I know for sure the pom is configured correctly in the workbench
Is the workbench actually calling "deploy" or just "install" ? Where in the code would be a good starting point to dig a little deeper on this?
From: rules-users-bounces(a)lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org> [mailto:rules-users-bounces@lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org>] On Behalf Of Joe White
Sent: Thursday, March 20, 2014 3:15 PM
To: Rules Users List
Subject: Re: [rules-users] Updating project repositories
I've got distribution management that matches my settings xml added to the pom but it doesn't seem to connect (or to attempt to connect)
<distributionManagement>
<repository>
<id>recondo-repo</id>
<url>http://papaafrepo001.recondo.vci:8081/artifactory/recondo-release</url<http://papaafrepo001.recondo.vci:8081/artifactory/recondo-release%3c/url>>
</repository>
<snapshotRepository>
<id>recondo-repo</id>
<url>http://papaafrepo001.recondo.vci:8081/artifactory/recondo-snapshot</url<http://papaafrepo001.recondo.vci:8081/artifactory/recondo-snapshot%3c/url>>
</snapshotRepository>
</distributionManagement>
From: rules-users-bounces(a)lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org> [mailto:rules-users-bounces@lists.jboss.org] On Behalf Of Michael Anstis
Sent: Thursday, March 20, 2014 2:56 PM
To: Rules Users List
Subject: Re: [rules-users] Updating project repositories
That depends how you're configuring your project's distribution management.
Adding (I assume) your remote to the repositories section is only telling maven a remote to query to *download* artifacts.
Sent on the move
On 20 Mar 2014 20:41, "Joe White" <Joe.White(a)recondotech.com<mailto:Joe.White@recondotech.com>> wrote:
So if I change the configuration in the WB UI should I expect the jar to end up in my remote repo? It is only ending up in my local repository.
From: rules-users-bounces(a)lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org> [mailto:rules-users-bounces@lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org>] On Behalf Of Michael Anstis
Sent: Thursday, March 20, 2014 1:10 PM
To: Rules Users List
Subject: Re: [rules-users] Updating project repositories
I've asked a colleague to comment.
We do manipulate the pom for the UI but should retain original content that is not modified by the Project Editor.
@Toni... What's your opinion?
Sent on the move
On 20 Mar 2014 18:03, "Joe White" <Joe.White(a)recondotech.com<mailto:Joe.White@recondotech.com>> wrote:
Also, that is the same configuration that is in my settings.xml for my installed maven instance and the profile is active.
From: rules-users-bounces(a)lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org> [mailto:rules-users-bounces@lists.jboss.org<mailto:rules-users-bounces@lists.jboss.org>] On Behalf Of Joe White
Sent: Thursday, March 20, 2014 12:01 PM
To: Rules Users List
Subject: [rules-users] Updating project repositories
I have updated my repository configuration but when I build and deploy by package two things happen:
- The jar only gets deployed to my local repo it doesn't go out to the remote instance
- The pom doesn't contain the full repository configuration when deployed. The <snapshot> and <releases> config are dropped
Am I missing a piece of config to get the jar pushed out to my remote repo?
My repository config inside the WB:
<repositories>
<repository>
<id>recondo-repo</id>
<name>papaafrepo001-releases</name>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>fail</checksumPolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>fail</checksumPolicy>
</snapshots>
<url>http://papaafrepo001.recondo.vci:8081/artifactory/repo</url<http://papaafrepo001.recondo.vci:8081/artifactory/repo%3c/url>>
</repository>
</repositories>
Repo config inside the deployed pom:
<repositories>
<repository>
<id>recondo-repo</id>
<name>papaafrepo001-releases</name>
<url>http://papaafrepo001.recondo.vci:8081/artifactory/repo</url<http://papaafrepo001.recondo.vci:8081/artifactory/repo%3c/url>>
</repository>
</repositories>
Joe White
CIO Technology Enabled Services
303-974-2849<tel:303-974-2849> (Office)
720-232-9023<tel:720-232-9023> (Cell)
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
10 years, 9 months
part time intern @ Red Hat
by Mark Proctor
I have a small budget for a part time intern for one year, 12K USD. The work can be 100% remote, from almost any country. This will mostly be UI work, improving our workbench.
If this interests you, email me off list.
Mark
10 years, 9 months
DummyKieScanner instantiated in catch{} in Impl
by Joe White
In the KieRepositoryImpl code what is the purpose of instantiating to the DummyScanner if the kie-ci jar is not on the classpath. Lines 120-126 in the impl. Ultimately this results in a Not Found exception downstream but masks the true issue of the missing jar. Seems like it would be better off throwing the exception back out to the caller so they know they are missing the jar?
I'm just trying to get a handle on how it is all wired together and spent a couple of hours chasing why it wouldn't find my kmodule jar when the core problem was my ear was just missing the kie-ci.
Thanks,
Joe
10 years, 10 months
Trying to develop RuleML (LegalRuleML) XML language for Drools
by Tom Rhodes
Hi!
I am trying to leverage/integrate the XML knowledge bases and rules with more mainstream programming tools and environments, like Drools. And therefore it would be nice to know the state-of-the-art of XML rule language in Drools.
As far as I understand, that up to the version 5.3 Drools supported 2 external languages (XML and DRL) that were mapped to the internal language, then the support for XML was dropped. Here is nice discussion about possible XML rule language and Drools:
http://drools.46999.n3.nabble.com/Enhancements-to-Rule-XML-BRL-td1082068....
I have several question regarding XML rule language:
1) what is the development status of it, specifically - does Drools really support standart RuleML? Here http://wiki.ruleml.org/index.php/RuleML_Implementations Drools is cited as the implementaton of RuleML but is is really so? At least Drools documentation does not say so. There is third-party translation tool available http://docs.codehaus.org/display/LOGICABYSS/RuleML2DroolsTranslator but is this sufficient for using RuleML with Drools?
2) Are there any suggestions or ideas for the implementation of RuleML translation to Drools internal language? Maybe it can be done in a straigtforward manner, but I feel that there should be some more formal solution. I.e. XML rule languages have formal syntax and apparently Drools internal language has formal syntax as well. If we could endow those languages with the formal semantics as well then the translation could be done in meta-level. I.e., maybe there are tools that can help to specify formal operational semantics for the formally given programming languages and then provide support for the translation in at this - more general level?
I would be glad to hear any suggestions or ideas about this.
Thanks!
Tom
10 years, 10 months
Guvnor Drools 5.6.Final : Not working dynamic update of knowledgebase when deploy latest rules using guvnor
by kunal
Same code is working when using Guvnor Drools 5.5, after I updated to drools
5.6.Final it stopped working of runtime update of knowledgebase.
KnowledgeAgentConfiguration kaconf =
KnowledgeAgentFactory.newKnowledgeAgentConfiguration();
kaconf.setProperty( "drools.agent.scanDirectories","false" );
KnowledgeAgent kagent = KnowledgeAgentFactory.newKnowledgeAgent( name,
kaconf );
// moniter KnowledgeAgent
kagent.monitorResourceChangeEvents(true);
kagent.addEventListener(new KBaseChangeEventListener());
kagent.applyChangeSet(
ResourceFactory.newClassPathResource("META-INF/ChangeSet.xml") );
startResourceChangeScannerService();
startResourceChangeNotifierService();
kbase = kAgent.getKnowledgeBase();
Please help me.
--
View this message in context: http://drools.46999.n3.nabble.com/Guvnor-Drools-5-6-Final-Not-working-dyn...
Sent from the Drools: Developer (committer) mailing list mailing list archive at Nabble.com.
10 years, 10 months
