[hibernate-dev] Query handling : Antlr 3 versus Antlr 4

[Steve Ebersole]

In the re-write case we will have to decide between 2 courses.

The problem lies in the fact that the listeners and visitors expect a tree
specifically generated from the grammar that generated them.  For HQL, say
we have one grammar named HqlParser (like in the poc).  The listeners and
visitors built from HqlParser specifically expect the tree from HqlParser.
The trees are typed.  Attempting to use trees from one grammar in the
listener/visitor from another grammar will not work.

As far as re-writing that effectively means 2 options.

If we want to have a second grammar for the "semantic query" we are going
to have to re-write the entire tree based on that second grammar.  I had
thought we might be able to mix them. But that was based on my Antlr 2/3
understanding where the trees are de-typed.  That approach will not work in
Antlr 4.  Its not a huge deal, but worth mentioning.

The other option would be to encode the "semantically correct" rules into
the original grammar (HqlParser) as a higher precedence than their parse
tree corollary.  This gets a little fugly.  Consider again the `select
c.headquarters.state.code` fragment.  During the parse phase we need to
accept any dotIdentifierPath as a selectable item.  We simply do not know
during parse what that represents.  So for the parse phase, a `selectItem`
rule (overly simplified) might look like:

selectItem : dotIdentifierPath;

In this approach we would re-write the tree "in place" during semantic
analysis.  So at some point we know that the given dotIdentifierPath
represents a reference to a persistent attribute.  So we'd alter that rule
to look contain alternatives for each semantic possibility:

selectItem : attributeReference | javaConstant | dotIdentifierPath;

The rules attributeReference and javaConstant would never match during the
parse phase.

Again, this is fugly imo.


On Wed, Jun 10, 2015 at 10:49 AM Gunnar Morling <gunnar at hibernate.org>
wrote:

> 2015-06-09 22:11 GMT+02:00 Steve Ebersole <steve at hibernate.org>:
>
>> So today I spent some time cleaning up the basic HQL parser.  Personally
>> I think it would be best if our 2 proof-of-concepts could share that first
>> grammar.  IMO that would make the differences between the 2 approaches more
>> apparent.  I will push those changes soon.
>>
>
> Ok, I can try and work on a PoC for the decorator-based approach. Not sure
> what the outcome will be, as your's, my understanding of it is roughly
> vague and high-level. But if it fails we can maybe settle for the Antlr3
> approach with the better feeling of having investigated the alternative.
>
> Can you let me know when you have pushed your stuff? What does it do,
> render the query below as SQL?
>
> It is not complete yet.  But it covers most cases.
>>
>>
>> On Tue, Jun 9, 2015 at 10:47 AM Steve Ebersole <steve at hibernate.org>
>> wrote:
>>
>>> On Tue, Jun 9, 2015 at 10:14 AM Gunnar Morling <gunnar at hibernate.org>
>>> wrote:
>>>
>>> Yes, indeed I cheated here a bit. Probably it should be the following
>>>> instead:
>>>>
>>>>       [DOT] ---> AttributeReference("<gen:1>", "code")
>>>>         [DOT]
>>>>           [DOT]
>>>>             [IDENT, "c"]
>>>>             [IDENT, "headquarters"]
>>>>           [IDENT, "state"]
>>>>         [IDENT, "code"]
>>>>
>>>
>>> How do you identify one DOT as referring to something else versus any of
>>> the other DOTs?
>>>
>>>
>>> Or maybe something like:
>>>>
>>>>     [SELECTION_PARTICLE] ---> AttributeReference("<gen:1>", "code")
>>>>       [DOT]
>>>>         [DOT]
>>>>           [DOT]
>>>>             [IDENT, "c"]
>>>>             [IDENT, "headquarters"]
>>>>           [IDENT, "state"]
>>>>         [IDENT, "code"]
>>>>
>>>> Where SELECTION_PARTICLE would be an abstract representation of
>>>> anything that can be selected (attribute ref, Java literal ref etc.) and
>>>> the decorator element added in a later pass would specify its actual
>>>> semantics based on the alias definitions etc. discovered before.
>>>>
>>>> Bottom line being, that decorators providing semantics are attached to
>>>> the nodes of the parse tree based on information gathered in previous
>>>> passes.
>>>>
>>>
>>> And what does that look like in real, practical terms?   That's what
>>> concerns me :)  I don't know, and you are just speaking in generalities.
>>> So what does that look like in practice?
>>>
>>>
>>> Not into the tree itself, but we can encode that semantic resolution
>>>> into decorators (node attachments).
>>>>
>>>
>>> Again, what do these "node attachments" look like in  practice?  I have
>>> zero clue and based on my discussions with Antlr folks its not pretty.
>>> Maybe I misunderstand.  But if you are proposing this approach, I would
>>> think you should have an idea of how it would look practically-speaking :)
>>>  Maybe this is the way to go, I just need to see what this looks like.
>>>
>>>
>>> Yes, they would deal with [[DOT][IDENT]] nodes but would benefit from
>>>> semantic decorators attached previously. During rendering I would expect
>>>> mainly those attachments to be of importance for the query creation.
>>>>
>>>> Admittedly, that's all quite "high level", but so far it seems doable
>>>> to me in principle. It doesn't answer of course actual tree transformations
>>>> such as (x + 0) -> x. I am not sure whether there are cases like this.
>>>>
>>>
>>> Yes it is all extremely high-level.  That is my concern.  Principle and
>>> practice are often 2 very different things.
>>>
>>> I plan on spending some time taking my hibernate-antlr4-poc project and
>>> expanding it specifically to try the "second grammar" approach and see what
>>> practical difficulties that shakes out.  Would you be willing to do the
>>> same for this decorated approach?  Then we'd have concrete stuff to compare
>>> and base a decision on.
>>>
>>> Also, `(x + 0) -> x` is actually a quite simple case.   Ours is much
>>> more complicated.  In analyzing `c.headquarters.state.code` in the SELECT
>>> clause we need a few things to happen in a few different parts of the
>>> tree.  We need:
>>> 1) `c.headquarters.state` to be transformed into 2 "implicit joins" in
>>> the FROM clause
>>> 2) we need to replace `c.headquarters.state.code` as
>>> `{implicit-alias}.code` in the SELECT
>>> 3) register `c.headquarters` and `c.headquarters.state` as implicit
>>> join paths (additional implicit joins using these paths should re-use the
>>> same joins).
>>>
>>