May 2007 - jboss-jira - Jboss List Archives

[JBoss JIRA] Reopened: (JBRULES-255) Dynamic index priorization

by Edson Tirelli (JIRA)

[ http://jira.jboss.com/jira/browse/JBRULES-255?page=all ] Edson Tirelli reopened JBRULES-255: ----------------------------------- > Dynamic index priorization > -------------------------- > > Key: JBRULES-255 > URL: http://jira.jboss.com/jira/browse/JBRULES-255 > Project: JBoss Rules > Issue Type: Feature Request > Security Level: Public(Everyone can see) > Components: Reteoo > Reporter: Edson Tirelli > Assigned To: Edson Tirelli > Priority: Minor > > All, > Follows some thoughts about the current and future solutions for beta node memory indexing. Suggestions and opinions are welcome. Sorry for the long e-mail. > I think a good way to discuss that is making an analogy to databases systems. As we all know, indexing is a great mechanism for performance improvements on database queries, but also represent a general overhead on other operations like insert, updates and deletes. Also, there is a memory consumption cost involved. A well planned set of indexes is essential for most enterprise applications and the responsibles for defining them are usually the DBAs. Once indexes are defined, when a query is executed against that database, a query planner component is used by database systems to estimate the best plan to run the query with the best performance, sometimes using the index, sometimes not. > Well, working memory has the same issues and same thoughts are valid here. Drools 3 implemented an automatic indexing strategy to index beta node memories in order to boost performance. Just to have some data to understand the consequences of it, lets use Manners 64 benchmark test results on a Pentium IV 3Ghz HT machine with 1,0 Gb memory. This is not really a detailed benchmark test, but simply some rough numbers in order to every one understand what happened, what is happening, and possibly suggest some paths for the future. (times in milliseconds) > Manners 64 without indexes: 135000 millisec to run > Manners 64 with beta node indexes: 10500 millisec to run > So, I think it is obviouse that indexes overall benefits pays off the overhead to keep them, at least in terms of performance. I'm not discussing limited memory enviroments here. > During the last few days I started to play with indexing again and find some odd results. To understand them, let me first explain that a beta node memory might be indexed by one or more indexes. It all depends on how many BoundVariableConstraints that beta node will have. For instance: > MyClass1( $myVal1 : attr1, $myVal2 : attr2 ) > MyClass2( attr1 == $myVal1, attr2 != $myVal2, attr3 == "bla", attr4 == 5 ) > The above rule would have a join node with two indexed constraints: attr1 == $myVal1, attr2 != $myVal2. > What happens is that the engine is capable of indexing none of them, one of them, or both of them. The way it is today, it will always index by both of them. > Just to investigate what are the indexing effects, I tried to make a code change to turn off multi-index for right beta memories. So, if a beta node has more than one constraint, I would elect one (arbitrarily) and index only that column. The result: > Manners 64 with indexing without multi-index on RIGHT memories: 12000 millisec to run > So, multi-index on right memories are good for manners. Then I turned on multi-index for right memories and turned off multi-index for left memories: > Manners 64 with indexing without multi-index on LEFT memories: 9500 millisec to run > So, basically, keeping multi-indexes for left memory on manners does not pay off the overhead. > As you can realize by now, this is TOTALLY dependent on rules AND data asserted into memory. For a different ruleset, the results could be totally different. > Besides that, somedays ago, a drools user reported an odd problem, that some may remember. He wrote a rule with the following column: > 1. alarmdefine(alarmreason==reason, alarmlevel!=level) > The rule was running very slowly for an average dataset (about 20 seconds). I sugested inverting the constraints to: > 2. alarmdefine(alarmlevel!=level, alarmreason==reason) > And the rule ran very fast (about 0.6 seconds). And the reason for this is because Drools has a fixed index ordering from last to first. So, for the first case above, where two indexes will be used (alarmreason and alarmlevel), drools will index the join node memory by [alarmlevel + alarmreason]. Nothing wrong so far, except that the data asserted into memory had all the same alarmlevel, so the index was adding overhead, but was completelly useless, as it was indexing the same value in all objects. (for more details on this subject, please look into list archive). > Thinking about that, I devised a strategy to do dynamic indexing priorization. The idea is that at run time, when a new object or tuple reaches a beta node and looks for matches, the engine will dynamically elect and order indexes to that specific match. So in the above case, it would always elect alarmreason as the primary index, but if someone asserts a fact where the alarmlevel becomes the most restrictive index, for that instance only, the primary index would be alarmlevel. > The result is that the rule runs in about 0.8 seconds for all cases, does not matter the way it is written. > Manners 64 with indexing priorization: 12000 millisec > As you can see, this dynamic indexing priorization has an overhead cost that allows one to not worry about how it is writing the rule, as long as he can accept a slowdown on performance. > Thinking in all that, I see only 2 ways for drools to follow: > 1. The first way is to allow users to fine tune their rules for best performance. This can be achieved making indexes as fast as possible and allowing the user to enable/disable/prioritize individual indexes on their rules. This is easier to implement, as only some syntax sugar needs to be supported by the parser, and the index factory needs access to the user definitions. > 2. The second way would be to develop a so smart optimizer that it can decide when to use and when not to use indexes as well as which index to use in a similar way some databases do.The index priorization strategy I developed might be a rough start, but it is miles away from being the solution. > Well, this is how things are on this area of development. My suggestion is, based on the results, not commit the index priorization stuff. I will just make a backup of it for future reference, but right now I think the overhead does not pay off. Better to educate users on how to write better rules for now. > If anyone has or knows about good material on this area of knowledge, I would really appreciate links or suggestions. > Let me know what you think. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

1
0
0 / 0

[JBoss JIRA] Closed: (JBRULES-190) "end" keyword in LHS, other keyword issues

by Mark Proctor (JIRA)

[ http://jira.jboss.com/jira/browse/JBRULES-190?page=all ] Mark Proctor closed JBRULES-190. -------------------------------- Fix Version/s: (was: 4.0.0.MR3) Resolution: Duplicate Issue > "end" keyword in LHS, other keyword issues > ------------------------------------------ > > Key: JBRULES-190 > URL: http://jira.jboss.com/jira/browse/JBRULES-190 > Project: JBoss Rules > Issue Type: Bug > Security Level: Public(Everyone can see) > Affects Versions: 3.0-rc1 > Reporter: Michael Neale > Assigned To: Bob McWhirter > > Need to refine drl.g to look (possibly) for EOLs when looking for "when", "end" etc etc.. (or optional ":" if one line) to stop parse errors for embedded keywords in java source etc etc. > This will require the multiphase parser mentioned elsewhere, as the general solution. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

1
0
0 / 0

[JBoss JIRA] Reopened: (JBRULES-190) "end" keyword in LHS, other keyword issues

by Mark Proctor (JIRA)

[ http://jira.jboss.com/jira/browse/JBRULES-190?page=all ] Mark Proctor reopened JBRULES-190: ---------------------------------- > "end" keyword in LHS, other keyword issues > ------------------------------------------ > > Key: JBRULES-190 > URL: http://jira.jboss.com/jira/browse/JBRULES-190 > Project: JBoss Rules > Issue Type: Bug > Security Level: Public(Everyone can see) > Affects Versions: 3.0-rc1 > Reporter: Michael Neale > Assigned To: Bob McWhirter > > Need to refine drl.g to look (possibly) for EOLs when looking for "when", "end" etc etc.. (or optional ":" if one line) to stop parse errors for embedded keywords in java source etc etc. > This will require the multiphase parser mentioned elsewhere, as the general solution. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

1
0
0 / 0

[JBoss JIRA] Created: (JBRULES-798) CLONE -null pointer with number comparison..

by Sanjay Mahurpawar (JIRA)

CLONE -null pointer with number comparison.. -------------------------------------------- Key: JBRULES-798 URL: http://jira.jboss.com/jira/browse/JBRULES-798 Project: JBoss Rules Issue Type: Bug Security Level: Public (Everyone can see) Affects Versions: 3.0.4 Environment: Windows XP or Solaris Sun JDK 1.4.2 Drools 3.0.4 distribution Reporter: Sanjay Mahurpawar Assigned To: Mark Proctor getting a null pointer while asserting fact: Caused by: java.lang.NullPointerException at org.drools.base.evaluators.IntegerFactory$IntegerGreaterEvaluator.evaluate(Unknown Source) at org.drools.rule.BoundVariableConstraint.isAllowed(Unknown Source) at org.drools.common.BetaNodeBinder.isAllowed(Unknown Source) at org.drools.reteoo.TupleSource.attemptJoin(Unknown Source) at org.drools.reteoo.JoinNode.assertTuple(Unknown Source) at org.drools.reteoo.LeftInputAdapterNode.createAndAssertTuple(Unknown Source) at org.drools.reteoo.LeftInputAdapterNode.assertObject(Unknown Source) at org.drools.reteoo.ObjectSource.propagateAssertObject(Unknown Source) at org.drools.reteoo.ObjectTypeNode.assertObject(Unknown Source) at org.drools.reteoo.Rete.assertObject(Unknown Source) at org.drools.reteoo.ReteooRuleBase.assertObject(Unknown Source) at org.drools.reteoo.ReteooWorkingMemory.doAssertObject(Unknown Source) at org.drools.common.AbstractWorkingMemory.assertObject(Unknown Source) at org.drools.common.AbstractWorkingMemory.assertObject(Unknown Source) Rule is something like this: rule "Greater Than with NULL number!!" when MyFact ( integerAttribute > 10 ) then ..... end the *intergerAttribute* is of type java.lang.Integer and value is *null*. no such problem if expression changed to *integerAttribute == 10* or != -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

2
4
0 / 0

[JBoss JIRA] Created: (JBRULES-784) NPE when left or right hand fact attribute is null

by David Nelson (JIRA)

NPE when left or right hand fact attribute is null -------------------------------------------------- Key: JBRULES-784 URL: http://jira.jboss.com/jira/browse/JBRULES-784 Project: JBoss Rules Issue Type: Bug Security Level: Public (Everyone can see) Components: Conflict Resolution Affects Versions: 3.1-m1, 3.0.6 Environment: Windows XP, JBoss 4.0.5, Java 1.5 Reporter: David Nelson Assigned To: Mark Proctor When a rule is evaluated, if the left or right side of a comparison is null, then the rule throws a NPE during execution which bubbles to the UI. Here's the simple scenario: Class A has a name attribute. Class B has a name attribute set to "Bob". when b : B() a: A( $name : name, ($name != b.getName()) ) then System.out.println("This is not an issue!"); -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

2
2
0 / 0

[JBoss JIRA] Updated: (JBRULES-231) add "not contains" constraint

by Edson Tirelli (JIRA)

[ http://jira.jboss.com/jira/browse/JBRULES-231?page=all ] Edson Tirelli updated JBRULES-231: ---------------------------------- Fix Version/s: 4.0.0.MR3 (was: 4.0.0.MR2) > add "not contains" constraint > ----------------------------- > > Key: JBRULES-231 > URL: http://jira.jboss.com/jira/browse/JBRULES-231 > Project: JBoss Rules > Issue Type: Feature Request > Security Level: Public(Everyone can see) > Components: Drl Parser/Builder > Reporter: Michael Neale > Assigned To: Edson Tirelli > Fix For: 4.0.0.MR3 > > > Person(addresses not_contains foo) > simple extension to the contains feature for collections. > Any other sytax ideas? > Person(something !contains foo) > Person(something !<< foo) > Person(something << foo) > I prefer as natural as possible... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

1
0
0 / 0

[JBoss JIRA] Created: (SECURITY-23) NullPointerException in JaasSecurityManager.getPrincipal()

by Alex Besogonov (JIRA)

NullPointerException in JaasSecurityManager.getPrincipal() ---------------------------------------------------------- Key: SECURITY-23 URL: http://jira.jboss.com/jira/browse/SECURITY-23 Project: JBoss Security Issue Type: Bug Security Level: Public (Everyone can see) Reporter: Alex Besogonov Assigned To: Anil Saldhana JaasSecurityManager.getPrincipal() does not check for empty domainCache. This method is called, for example, when JAAS is used from web layer. Trivial fix, add in the start of the method: if( domainCache == null ) return principal; -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

2
5
0 / 0

[JBoss JIRA] Assigned: (JBRULES-231) add "not contains" constraint

by Michael Neale (JIRA)

[ http://jira.jboss.com/jira/browse/JBRULES-231?page=all ] Michael Neale reassigned JBRULES-231: ------------------------------------- Assignee: Edson Tirelli (was: Michael Neale) > add "not contains" constraint > ----------------------------- > > Key: JBRULES-231 > URL: http://jira.jboss.com/jira/browse/JBRULES-231 > Project: JBoss Rules > Issue Type: Feature Request > Security Level: Public(Everyone can see) > Components: Drl Parser/Builder > Reporter: Michael Neale > Assigned To: Edson Tirelli > Fix For: 4.0.0.MR2 > > > Person(addresses not_contains foo) > simple extension to the contains feature for collections. > Any other sytax ideas? > Person(something !contains foo) > Person(something !<< foo) > Person(something << foo) > I prefer as natural as possible... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

1
0
0 / 0

[JBoss JIRA] Created: (JBRULES-577) Please add "contained by" syntax for constraints

by Dirk Bergstrom (JIRA)

Please add "contained by" syntax for constraints ------------------------------------------------ Key: JBRULES-577 URL: http://jira.jboss.com/jira/browse/JBRULES-577 Project: JBoss Rules Issue Type: Feature Request Security Level: Public (Everyone can see) Components: Drl Parser/Builder Reporter: Dirk Bergstrom Assigned To: Mark Proctor I'd like to be able to write a constraint to determine if a field's value is a member of some collection. This is the inverse of the "contains" keyword. The proposed syntax is: [bindingVar :] <fieldName> in <someCollection> (I'm not picky about the keyword, other possibilities are "containedBy" or "memberOf", but "in" is a single word) Use case: I have a container class with a field that exposes a collection of record numbers. In the working memory, I have a bunch of record objects, and one container object. I want to write a rule that fires when it encounters a container that has records matching some criteria: when Container( $recordNums : records ) Record( number in $recordNums, otherField == "somevalue" ) then doStuff(); The current workaround is: Record( num : number -> ( $recordNums.contains(num) ) ) Which is problematic, as the predicate may not be time-constant. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

5
7
0 / 0

[JBoss JIRA] Reopened: (JBRULES-231) add "not contains" constraint

by Edson Tirelli (JIRA)

[ http://jira.jboss.com/jira/browse/JBRULES-231?page=all ] Edson Tirelli reopened JBRULES-231: ----------------------------------- "not contains" was not implemented, since we have "excludes". Do you want me to add "not contains" as a synonym for "excludes"? > add "not contains" constraint > ----------------------------- > > Key: JBRULES-231 > URL: http://jira.jboss.com/jira/browse/JBRULES-231 > Project: JBoss Rules > Issue Type: Feature Request > Security Level: Public(Everyone can see) > Components: Drl Parser/Builder > Reporter: Michael Neale > Assigned To: Michael Neale > Fix For: 4.0.0.MR2 > > > Person(addresses not_contains foo) > simple extension to the contains feature for collections. > Any other sytax ideas? > Person(something !contains foo) > Person(something !<< foo) > Person(something << foo) > I prefer as natural as possible... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

17 years, 8 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

jboss-jira May 2007