Left and Right Unlinking - Community Project Proposal
by Mark Proctor
In an effort to help encourage those thinking of learning more about
the internals of rule engines. I have made a document on implementating
left and right unlinking. I describe the initial paper in terms relevant
to Drools users, and then how that can be implemented in Drools and a
series of enhancements over the original paper. The task is actually
surprisingly simple and you only need to learn a very small part of the
Drools implementation to do it, as such it's a great getting started
task. For really large stateful systems of hundreds or even thousands of
rules and hundreds of thousands of facts it should save significant
amounts of memory.
http://blog.athico.com/2010/08/left-and-right-unlinking-community.html
Any takers?
Mark
Introduction
The following paper describes Left and Right unlinking enhancements for
Rete based networks:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.45.6246
<http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.45.6246>
A rete based rule engine consists of two parts of the network, the alpha
nodes and the beta nodes. When an object is first inserted into the
engine it is discriminated against by the object type Node, this is a
one input and one output node. From there it may be further
discriminated against by alpha nodes that constrain on literal values
before reaching the right input of a join node in the beta part of the
network. Join nodes have two inputs, left and right. The right input
receives propagations consisting of a single object from the alpha part
of the network. The left input receives propagations consisting of 1 or
more objects, from the parent beta node. We refer to these propagating
objects as LeftTuple and RightTuple, other engines also use the terms
tokens or partial matches. When a tuple propagation reaches a left or
right input it's stored in that inputs memory and it attempts to join
with all possible tuples on the opposite side. If there are no tuples on
the opposite side then no join can happen and the tuple just waits in
the node's memory until a propagation from the opposite side attempts to
join with it. If a given. It would be better if the engine could avoid
populating that node's memory until both sides have tuples. Left and
right unlinking are solutions to this problem.
The paper proposes that a node can either be left unlinked or right
unlinked, but not both, as then the rule would be completely
disconnected from the network. Unlinking an input means that it will not
receive any propagations and that the node's memory for that input is
not populated, saving memory space. When the opposite side, which is
still linked, receives a propagation the unlinked side is linked back in
and receives all the none propagated tuples. As both sides cannot be
unlinked, the paper describes a simple heuristic for choosing which side
to unlink. Which ever side becomes empty first, then unlink the other.
It says that on start up just arbitrarily chose to unlink one side as
default. The initial hit from choosing the wrong side will be
negligible, as the heuristic corrects this after the first set of
propagations.
If the left input becomes empty the right input is unlink, thus clearing
the right input's memory too. The moment the left input receives a
propagation it re-attaches the right input fully populating it's memory.
The node can then attempt joins as normal. Vice-versa if the right input
becomes empty it unlinks the left input. The moment the right input
receives a propagation it re-attaches the left input fully populating
it's memory so that the node can attempt to join as normal.
Implementing Left and Right Unlinking for shared Knowledge Bases
The description of unlinking in the paper won't work for Drools or for
other rule engines that share the knowledge base between multiple
sessions. In Drools the session data is decoupled from the main
knowledge base and multiple sessions can share the same knowledge base.
The paper above describes systems where the session data is tightly
coupled to the knowledge base and the knowledge base has only a single
session. In shared systems a node input that is empty for one session
might not be empty for another. Instead of physically unlinking the
nodes, as described in the paper, an integer value can be used on the
session's node memory that indicates if the node is unlinked for left,
right or both inputs. When the propagating node attempts to propagate
instead of just creating a left or right tuple and pushing it into the
node. It'll first retrieve the node's memory and only create the tuple
and propagate if it's linked.
This is great as it also avoids creating tuple objects that would just
be discarded afterwards as there would be nothing to join with, making
things lighter on the GC. However it means the engine looks up the node
memory twice, once before propagating to the node and also inside of the
node as it attempt to do joins. Instead the node memory should be looked
up once, prior to propagating and then passed as an argument, avoiding
the double lookup.
Traditional Rete has memory per alpha node, for each literal constraint,
in the network. Drools does not have alpha memory, instead facts are
pulled from the object type node. This means that facts may needlessly
evaluate in the alpha part of the network, only to be refused addition
to the node memory afterwards. Rete supports something called "node
sharing", where multiple rules with similar constructs use the same
nodes in the network. For this reason shared nodes cannot easily be
unlinked. As a compromise when the alpha node is no longer shared, the
network can do a node memory lookup, prior to doing the evaluation and
check if that section of the network is unlinked and avoid attempting
the evaluation if it is. This allows for left and right unlinking to be
used in a engine such as Drools.
Using Left and Right Unlinking at the Same Time
The original paper describes an implantation in which a node cannot have
both the left and right inputs unlinked for the same node. Building on
the extension above to allow unlinking to work with a shared knowledge
base the initial linking status value can be set to both left and right
being unlinked. However in this initial state, where both sides are
unlinked, the leaf node's right input isn't just waiting for a left
propagation so the right can re-link itself (which it can't as the left
is unlinked too). It's also waiting to receive it's first propagation,
when it does it will link the left input back in. This will then tell
it's parent node's right input to also do the same, i.e. wait for it's
first right input propagation and link in the left when it happens. If
it already has a right propagation it'll just link in the left anyway.
This will trickle up until the root is finally linked in and
propagations can happen as normally, and the rule's nodes return to the
above heuristics for when to link and unlink the nodes.
Avoid Unnecessary Eager Propagations
A rule always eagerly propagates all joins, regardless of whether the
child node can undertake joins too, for instance of there is no
propagates for the leaf node then no rules can fire, and the eager
propagations are wasted work. Unlinking can be extended to try to
prevent some level of eager propagations. Should the leaf node become
right unlinked and that right input also become empty it will unlink the
left too (so both sides are unlinked) and go back to waiting for the
first right propagation, at which point it'll re-link the left. If the
parent node also has it's right input unlinked at the point that it's
child node unlinks the left it will do this too. It will repeat this up
the chain until it reaches a node that has both left and right linked
in. This stops any further eager matching from occurring that we know
can't result in an activation until the leaf node has at least one right
input.
Heuristics to Avoid Churn from Excessive and Unnecessary Unlinking
The only case where left and right linking would be a bad idea is in
situations that would cause a "churn". Churn is when a node with have a
large amount of right input memory is continually caused to be linked in
and linked out, forcing those nodes to be repeatedly populated which
causes a slow down. However heuristics can be used here too, to avoid
unnecessary unlinking. The first time an input becomes empty unlink the
opposite and store a time stamp (integer counter for fact handles from
the WM). Then have a minimum delta number, say 100. The next time it
attempts to unlink, calculate the delta of the current time stamp
(integer counter on fact handle) and the time stamp of the node which
last unlinked (which was recorded at the point of unlinking) if it's
less than 100 then do nothing and don't unlink until it's 100 or more.
If it's 100 or more then unlink and as well as storing the unlink time
stamp, then take the delta of 100 or more and apply a multiple (2, 3, 4
etc depending on how steep you want it to rise, 3 is a good starting
number) and store it. Such as if the delta is 100 then store 300. The
next time the node links and attempts to unlink it must be a delta of
300 or more, the time after that 900 the time after that 2700.
14 years
Multi-user architecture of Drools Web Service.
by tom ska
Hi,
what I have is SOAP Web Service with two methods: fn_AddFacts, and
fn_Conclude. I defined special XML implementation to send various fact's
types via SOAP. Model is defined in DRL/Guvnor - and rules too.
So now, I can add some facts (web service creates them using Drools
"reflection" API) to knowledge base. And then use fn_Conclude method, to
fire "fireAllRules" method and get response with results. But.....
What if now I have 100 users, and I don't want their's facts to interfere
each other? I want, to use Drools to conclude for different users. I want to
use this same rules, but on different knowledge bases (each user has own
knowledge base of his facts).
Please help me, how to solve this problem... I am new in JAVA EE, and I
don't understand some elementary issues well. (But I am trying to understand
them :D )
regards,
tom.
14 years
error while inserting values in excel
by Kripa Nathwani
Hi,
Below is the decision table on which I am working. It is calculating PF and HRA by taking basic value and checking that basic is not equal to null.
The problem is when I start inserting values in the column it is giving me following error messages:
no viable alternative at input ')' in rule "Pricing bracket_12" in pattern AmountPojo
mismatched input '!=' expecting ')' in rule "Pricing bracket_12" in pattern AmountPojo
Unknown error while parsing. This is a bug. Please contact the Development team
I know it is related to syntax but I am not able to solve it.
[cid:image004.png@01CB404C.DD13B6E0]
Best Regards,
Kripa
________________________________
This Email may contain confidential or privileged information for the intended recipient (s) If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system.
______________________________________________________________________
14 years
Time unit step-size inside rule files.
by Thorsten
Hello out there,
is it possible to change or extend the existing time unit step-size like
[h,s,ms] used inside the rules? They all seems to have a fixed (logical)
relationship (e.g. 1 s = 1000ms). Unfortunately we have a different time
representation in our application. We can rebuild the time model using
the smallest units as pseudo time reference but this makes the rules
hard to write / read as every user has to know how much
“smallest-time-units” represents a real world second.
Thank you very much
Thorsten
14 years
Skills required for using Drools Guvnor
by Swapnil Sawant
Hi,
I had a very basic question. I wanted to know the pre-requisite skills which are required in order to start working on drools guvnor GUI.
When I say work , I mean creating rules/modifying them etc.
Technical knowledge(e.g. java technology) is must in this case?
Thanks & Regards,
Swapnil Sawant
S
________________________________
This Email may contain confidential or privileged information for the intended recipient (s) If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system.
______________________________________________________________________
14 years
Drools beginners guide
by Singh, Palvinder X.
I am new to Drools and just executed with my first sample program. I
have been trying to find out some beginners guide to help me in start
up, but unable to find any over the net. I would appreciate if anyone
can share some material to go thru.
Just to set up some background, I am not new to IT but yes new to
Business rule engine.
Thanks,
Palvinder Singh
14 years
NPE in 5.1 PackageBuilder due to misspelled enum name
by Wolfgang Laun
Given an enum Fruit { CHERRY, STRAWBERRY } a rule with a typo in an enum name
Jam( fruit == Fruit.CHERIE || == Fruit.STRAWBERRY,... )
causes the NPE shown below while adding the DRL resource:
Exception in thread "main" java.lang.NullPointerException
at org.drools.rule.AbstractCompositeRestriction.getRequiredDeclarations(AbstractCompositeRestriction.java:59)
at org.drools.rule.MultiRestrictionFieldConstraint.getRequiredDeclarations(MultiRestrictionFieldConstraint.java:73)
at org.drools.rule.Pattern.setConstraintType(Pattern.java:358)
at org.drools.rule.Pattern.addConstraint(Pattern.java:226)
at org.drools.rule.builder.PatternBuilder.build(PatternBuilder.java:432)
at org.drools.rule.builder.PatternBuilder.buildConstraint(PatternBuilder.java:264)
at org.drools.rule.builder.PatternBuilder.build(PatternBuilder.java:213)
at org.drools.rule.builder.PatternBuilder.build(PatternBuilder.java:108)
at org.drools.rule.builder.GroupElementBuilder.build(GroupElementBuilder.java:69)
at org.drools.rule.builder.RuleBuilder.build(RuleBuilder.java:79)
at org.drools.compiler.PackageBuilder.addRule(PackageBuilder.java:1151)
at org.drools.compiler.PackageBuilder.addPackage(PackageBuilder.java:637)
at org.drools.compiler.PackageBuilder.addPackageFromDrl(PackageBuilder.java:267)
at org.drools.compiler.PackageBuilder.addKnowledgeResource(PackageBuilder.java:459)
at org.drools.builder.impl.KnowledgeBuilderImpl.add(KnowledgeBuilderImpl.java:28)
at rss.checker.engine.impl.DroolsEngine.loadSourceRules(DroolsEngine.java:49)
at rss.checker.init.Main.exec(Main.java:36)
at rss.checker.init.Main.main(Main.java:63)
14 years
Gunvor Test Scenarios in Batch?
by Lawrence Terrill
I'd like to use Guvnor-defined test scenarios in a externally invoked 'batch' process in order to run all the tests for a given change-set or test package in an automated process rather than through the Guvnor GUI, such as when new Java code is checked in. Is there an API or other facility that supports such a thing?
Thanks!
Larry
14 years
Getting Exception firing rule second time with same KnowledgeBase
by gauravs
Hello,
I am facing this problem which I tried to resolve in all possible ways I
could think of but situation is bit crazy here. I am creation knowledgebase
object and putting it in servetCOntext object. On HTTP Request I create a
new session, insert facts and fire all the rules. Every thing works well
first time. But repeating the same (making another HTTP Request) throws
exception shown below. If I create fresh knowledgebase every time problem
doesn't show up at all.
Please suggest where I might be going wrong. Google suggest that somewhere
object are getting null. I verified that too. It doesn't seem to be the
case.
Regards,
-Gaurav.
org.drools.runtime.rule.ConsequenceException: [Error: unable to access
field]
[Near : {... Unknown ....}]
^
[Line: 1, Column: 0]
at
org.drools.runtime.rule.impl.DefaultConsequenceExceptionHandler.handleException(DefaultConsequenceExceptionHandle
r.java:23)
at
org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:943)
at
org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:885)
at
org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1086)
at
org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:660)
at
org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:627)
at
org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:183)
--
View this message in context: http://drools-java-rules-engine.46999.n3.nabble.com/Getting-Exception-fir...
Sent from the Drools - User mailing list archive at Nabble.com.
14 years
