[teiid-dev] [Teiid Designer] - Architecture for Access to multiple Teiid Runtimes

> Teiid has introduced a query syntax tree that it uses for > transformations. I think it difficult not > to make that available to clients wishing to create queries > consistent with teiid's syntax > (especially given that a partial api in the teiid-api library of the > syntax tree is already > provided).

The major hurdle in consolidating the two apis has been the Designer usage of the engine api.

The secondary issue is the build-up of logic on the engine api that has nothing to do with the needs of translator developers and should not be exposed via a public api.

Given Designer's willingness to massively change it's usage of the Teiid engine language objects we can revisit consolidating them on the Teiid side.

However I am not a fan of proliferating a lot of interfaces into Teiid at the language object level.

There is also the issue of the QueryMetadataInterface and the implementation of the metadata identifier object. I would be absolutely unwilling to expose that paradigm at the translator level which directly uses the metadata api instead. You would have to promote yet another set of interfaces for the metadata objects.

For Designer to eventually support better comment/whitespace preservation, incremental parsing, etc. some split in our implementations may be needed - otherwise you'd have to push that logic to our side (so we'd have to update our language objects to generally include comments, line numbers, make parser modifications, etc.).

An intermediate alternative step would be to move Designer off of our language objects and onto a more stable/lower level AST api (just a modified token chain out of JavaCC). You'd be able to reuse our parser and we'd have to think > our way though the other usage - for example the major hurdle with a token approach assuming that you are reusing our parser/resolving logic is associating the resolved object reference / type info with the AST position.

In any case I've never liked the interaction of placing a lot of designer/display concerns onto the Teiid side (such as requiring retaining parts of the parse string for display and the notion of modifying the display mode, etc.). At the least for the runtime, there can be an unnecessary increase in the memory footprint of the language objects.

> If designer was to use another syntax tree, it would be > impossible to ensure consistency > between the two syntaxes, (kinda like designer creating ruby and > teiid processing it as python).

That is not a good analogy. The eclipse tooling already obviously has lots of support for dealing with many languages - without the requirement of having a language api provided to them (which is true for Java support itself since JDT uses an Eclipse specific AST api). There is also quite a significant amount of code in the datatools area that attempts to provide Eclipse with SQL support. I guess my point could be summed up as perhaps we can look for a better long term decoupling rather than simply refining/codifying the current dependencies.

Hi Ramesh, Thought it better to continue this discussion on the mailing list ... See answers in line. > Do you know that these classes are Teiid internal classes and they > can change from version to > version? We have no intension of making these public as there is > little too much code. Completely agree which is why I would suggest these classes implement interfaces, eg. LanguageObject implements ILanguageObject The only inconsistency between teiid 7.7.1 and teiid 8.2 as far as the query syntax tree goes is that some classes have changed their hierarchy. The internals of the classes may have changed but that is uninteresting to client applications like designer. To illustrate I'll use the example of ElementSymbol: * 275 references in the designer codebase of IElementSymbol ** References in the teiid 8.2 plugin, providing the implementation, syntax factory implementation and class implementations of visitor interfaces ** References in the teiid 7.7.2 plugin, providing the implementation, syntax factory implementation and class implementations of visitor interfaces ** References as a method parameter, instanceof checks, casts from ILanguageObject or ISymbol, array declarations ** References in the spi interface plugin, which would disappear if included in teiid *** References in the interface IUpdateValidator where implementations wrap the teiid UpdateValidator since the latter uses classes rather than interfaces *** Other spi interfaces of query syntax objects ** Interface methods expected from IElementSymbol *** IGroupSymbol getGroupSymbol(); *** void setGroupSymbol(IGroupSymbol groupSymbol); *** boolean isExternalReference(); *** void setDisplayFullyQualified(boolean value); *** ESDisplayMode getDisplayMode(); *** void setType(Class<?> targetType); *** Object getMetadataID(); *** void setMetadataID(Object metadataID); ** References in tests so can be discounted Thus, designer expects an object that can be identified as an IElementSymbol, so it can be cast and provide a few methods. In a small number of cases a new IElementSymbol is expected to be created so a factory would be required. Some of the less common syntax objects are referenced less frequently, in fact only in visitors. In this case, implementation delegate classes would not created but a LanguageObjectImpl was instead wrapping the delegate. > When you say backport interfaces to Teiid, that amounts to same as > Teiid releasing the same as public api and > keep up with backwards compatibility. Teiid already defines another > set language objects in the public API, see the "teiid-api" > project, > however they are currently used only in the translator level. The > query engine and Designer use the > other internal copy. In long term, we hope to merge these two sets > of language objects into one that > are based on the "teiid-api" ones, but I do not expect that > happening any time real soon. Even though the 'teiid-api' objects have not been unified with their engine counterparts, there is nevertheless an expectation that objects of this nature will be part of the public api due to their inclusion in the teiid-api library. However, it is quite right that such classes are subject to change but unlikely that their method signatures would be modified to the extent that a set of consistent interfaces could not be located in the teiid-api library. Both sets of current syntax classes (teiid-api and engine versions) could implement the interfaces thereby beginning the process of unifying them. Since the internals of these classes remain unimportant to designer, their existence as a set of interfaces and a factory is all that is required. When I refer to 'backport interfaces to Teiid'. I mean that releases of teiid would be retrofitted with these interfaces to allow designer (and other applications that care) to have some access to the same query syntax tree that is being used by teiid. That way all client applications could understand and generate the same query language as the teiid server is expecting. The changes in hierarchy remain an issue. I have worked around it by adding isFunction() and isExpression() methods to the ILanguageObject interface. Instead of doing instanceof checks, eg. object instanceof IExpression, in Designer, these method are called instead with their teiid 7.7.2 and 8.2 implementations returning different answers. > > We have talked many times before about Designer forking these > necessary libraries or finding another > SQL language framework (DTP has one) to circumvent these issues. > So, in general we think this > feature more complicated than creating delegates and I do not > believe that is the right approach. > Not sure why this feature is complex given the provision of interfaces. Would you mind clarifying this further for me? Delegating classes is not a long term option but at this point, it was the only way of creating a class that implements a necessary interface but does not modify the teiid delegate class. Teiid has introduced a query syntax tree that it uses for transformations. I think it difficult not to make that available to clients wishing to create queries consistent with teiid's syntax (especially given that a partial api in the teiid-api library of the syntax tree is already provided). If designer was to use another syntax tree, it would be impossible to ensure consistency between the two syntaxes, (kinda like designer creating ruby and teiid processing it as python). As I am sure you know, it would be necessary for the teiid-api library with such interfaces to remain consistent and backward compatible between versions. Designer would only ever import the latest version of this api library, separately to the 7.7.2 and 8.2 client-runtime plugins, both of which would then depend on it. In this way, most of the delegating classes would disappear. Hopefully, that is not too opaque and sounds workable. I am sure further discussion is required and more issues need to be ironed out but at this point it seems that teiid only has to introduce these interfaces into the api and designer can remain compatible as well as support multiple versions of the client-runtime. Your further thoughts always welcome. PGR -- Paul Richardson