Fine grained Meta-Data driven intelligent parser ------------------------------------------------ Key: JBRULES-264 URL: http://jira.jboss.com/jira/browse/JBRULES-264 Project: JBoss Rules Issue Type: Feature Request Security Level: Public(Everyone can see) Components: Drl Parser/Builder Reporter: Mark Proctor Assigned To: Bob McWhirter Fix For: 4.0.0.MR3 Currently we use the same API to parse and compile packages as we do for the IDE. This is proving quite heavy. I propose that we make an additional API for IDE integration that is fine grained and meta data driven. I believe that Eclipse has extensive meta data capabilities, which it stores on disk, we should really try and leverage this. Although the core API should try and be independent, so other IDEs can leverage it. 1) Split the main document up into package, imports, globals, functions and rules ? but do not parse contents. a. Maybe we will build up the meta-data for imports and globals at this stage, as I imagine that?s easy. b. Record, as meta data, the start and end line numbers, i.e. the range, for all sections. 2) Tokenise all expressions and blocks ? do not loose line numbers though, so will need to pad. a. Record, as meta data, the start and end line/col numbers, i.e. the range, for all expressions and blocks. 3) Parse each Rule a. Validate Conditional Element and Field Constraint structure b. Validate Columns and Fields ? record this in meta data. So we know the dependant classes and fields for this rule. c. Record Column and Field bindings as meta data ? as used for expressions. d. Validate operators and the RHS value type, checks it?s valid with LHS and operator. 4) Determine required declarations for each expression and record as meta data a. If we can it might be nice to also determine dependant classes from the Imports and Globals , beyond declarations, and record as meta data. 5) Compile each expression and block, using a helper util, record the errors as meta data and then forget the compiled .class 6) Develop intelligent balanced-text recovery. So when scanning 1) and 2) we need to check for balanced text, if we detect incorrect balancing we then mark that section as invalid and find the start of the next valid section ? nothing inside those invalid sections will be parsed. a. i.e. if we have an invalid expression, incorrect number of brackets on the LHS of the rule, we try and recover to the next valid area, ideally this would be the next valid conditional element ? but that may be hard and it could be the RHS of the rule. Start simple, make it coarse, and intelligent fine grained can be added later. 7) Intelligent re-parsing for project wide changes. a. We have class dependencies in meta-data for the various sections ? and also fields in constraints. So we can determine errors from the meta data, without having to reparse. b. In the case of expressions we can use the meta data, to avoid re-compiling. I guess if the changes are too dramatic then we can recompile the expression/consequence. We have the ranges in meta data for the dependant sections, so we can rescan to suck up the expression, without having to parse the entire document c. In document editing we only recompile expressions consequence if the user edits them ? again we should know that we are in an expression, we know the start so we scan from the start of t he expression/block to the end and compile. Avoid reparse the entire document or even the rule. If bindings are changed we can also determine the dependant expressions and recompile to get errors. I am sure there is a lot more complexity to this and missed stages. But it should be enough to show you the direction I want to take this. So we are extensively using meta data to minimise the re-parsing and re-compiling. We are also using meta data to localise the areas that we do need to re-parse for changes. The key to this is always being able to know exactly what we are editing. We may have to extend the descr, or create a new structure to handle meta data driven parsing. Non of this replaces the existing ruleparse/descry/packagebuidler implementations which are still required for compiling and deploying real rule bases - although hopefully we can leverage parts from both, to avoid duplication.