[dna-issues] [JBoss JIRA] Commented: (DNA-467) Add search/query support to the graph API

[Randall Hauch (JIRA)]

    [ https://jira.jboss.org/jira/browse/DNA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12486721#action_12486721 ] 

Randall Hauch commented on DNA-467:
-----------------------------------

Added an initial version of the query engine functionality.  This design provides a way to define and supply queries (in a various languages) and have the engine parse them into a single Abstract Query Model (equivalent to the abstract syntax tree for a query), plan, validate, optimize and then process the portions atomic portions of the query plan.  This whole system was designed to be easily reused as-is or extended and customized to provide the desired behavior.  But because this is a generalized query engine capable of query over a 'graph', the actual processing of the atomic portions of the queries must be provided when the engine is used.  Part of this commit includes a new 'dna-search' project containing a specialization of the query engine with a processor capable of using a set of Lucene search indexes, along with utility and management methods to populate and update the indexes (by indexing the entire content and/or by updating the content based upon events).

A number of packages were added to 'dna-graph', including: an abstract query model (AQM) based upon the JSR-283 specification in 'o.j.dna.graph.query.model'; a query engine component in 'o.j.dna.graph.query', a query planning module in 'o.j.dna.graph.query.plan'; an extensible rule-based optimization module in 'o.j.dna.graph.query.optimize'; a simple way to define the schemata that is being queried in 'o.j.dna.query.validate'; a flexible processing plan model and execution framework in 'o.j.dna.graph.query.process'; and a framework for different query language parsers (including a JCR-SQL2 parser) in 'o.j.dna.query.parse'.  This entire query engine framework was designed to be reused and/or extended in multiple places, and so includes a way to accept and execute queries from a number of different an abstraction of the actual processing of the low-level atomic queries.  Numerous unit tests were added to test each of the components, including a large number of tests for the SQ!
 L parser.

A new 'dna-search' project was created and the initial Lucene-based query engine functionality was added.  Quite a few tests were added to verify the desired behavior.

At this point, the general query engine and the Lucene-based specialization are for the most part complete and thoroughly tested, but these components need to be integrated into the larger connector framework and JCR implementation.  All of the Lucene index generation and management needs to be coordinated and integrated with the administration and lifecycle of the DNA connectors and JCR engine.  Additionally, while there are methods to create/update the indexes, the ability to extract text from binary property values still needs to be added. In short, there still is a lot of outstanding work.

> Add search/query support to the graph API
> -----------------------------------------
>
>                 Key: DNA-467
>                 URL: https://jira.jboss.org/jira/browse/DNA-467
>             Project: DNA
>          Issue Type: Feature Request
>          Components: API, Connectors, Graph
>    Affects Versions: 0.5
>            Reporter: Randall Hauch
>            Assignee: Randall Hauch
>            Priority: Critical
>             Fix For: 0.7
>
>
> The graph system needs to be able to push a search/query down to a connector, and we need to develop a query model (in the form of concrete classes, probably immutable).  Connectors can optionally support queries, and if they do they will process the query and return the results.
> Particular language bindings would be put on top of this query model.  For example, JCR 1.0 defines an XPath language.  Support for each language would parse a query in that language and produce a query model.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira