[teiid-dev] [teiid-designer-dev] Translator based Metadata

[Ramesh Reddy]

@Barry: yes and no. Technically both are source models, but the
translators in (1) provide more rich metadata, where as in (2) are
static models, are not that useful with out additional view based metadata.

Also, we do not need to replace what we have right away with new ones,
but add the new translator based import and move towards that goal to
convert all as time permits.

The basic idea is, if there is a running server, instead of Designer
making direct connection to the source we make use of server's
connection and get metadata. This can be true for jdbc, file,
web-service, salesforce etc. Designer already does this for preview, we
need to push for metadata too. Once the metadata is retrieved, any
additional stuff like view generation (ex: xml stuff) can be done on top.

Using the translator-api in Designer does not bother me that much as it
is public api. I know Barry was interested in hiding the physical models
for atleast the translators in group (2), but not sure how we can
accomplish with so much legacy code and design habits.

Ramesh..

On 04/26/2012 01:14 PM, Barry Lafond wrote:
> Importing into Designer means a couple of things
> 
> (1) Import XXXX and create Source Relational Model
> 
>   * JDBC Importer
>   * DDL Importer (though not really a "source" since it can't be queried)
>   * Salesforce Importer
> 
> 
> (2) Import XXXX and create a View Model with Tables/Procedures that
> query Sources.
> - There ARE source models created, but they are only required to define
> the source procedures (getTextFiles(), invoke(), etc)
> - We also use the source models to persist and transport the source
> connection/translator info from the model in designer to the VDB (needed
> for Preview Data too)
> 
>   * Flat File Importer
>   * XML File Importer
>   * WSDL Importer
> 
> So thinking that Translator-based Metadata would apply more directly to
> type (1) importers and not sure how applicable it would be to (2) since
> the "source" isn't really model, just the "views" of the data the user
> wishes to query.
> 
> Barry
> 
> ------------------------------------------------------------------------
> 
>     *From: *"Steven Hawkins" <shawkins at redhat.com>
>     *To: *"teiid-designer-dev" <teiid-designer-dev at lists.jboss.org>
>     *Cc: *teiid-dev at lists.jboss.org
>     *Sent: *Thursday, April 26, 2012 12:30:38 PM
>     *Subject: *Re: [teiid-designer-dev] [teiid-dev] Translator based
>     Metadata
> 
>     Moving over to designer dev.  There are several integration options
>     to get most of the importer code out of Designer:
> 
>     1. Reuse translator libraries in Designer (but increases code
>     dependency on Teiid).
>     2. Require a running server and use the dynamic vdb approach to
>     capture a physical model.
>       a. serialize the metadata objects for consumption by Designer
>     (same as 1).
>       b. as Ramesh suggests create a ddl dump for consumption by Designer.
> 
>     Either of the above approaches do provide a path toward a Designer
>     integration of custom Translators that does not also involve
>     necessarily coding a custom importer GUI.
> 
>     The hold ups for a redesign are that the import code, especially for
>     JDBC, is quite expansive and intricately tied to the GUI.  Also
>     Teiid Translators to not advertise their import properties and
>     Translators just provides a single getMetadata call that gets the
>     full form or all metadata.  Translators do not yet provide more
>     interrogative methods that are better fits in a multi-step wizard -
>     such as just providing schema/table/procedure names for multi-select
>     filtering prior to full import.  
> 
>     At one point it was also proposed to get rid of or hide physical
>     models.  I somewhat agree with that, however designer still needs to
>     display the source metadata in some view and there are still
>     situations that require hand-crafted metadata as a workaround (ldap,
>     using sequences, etc.).  Also for performance reasons the source
>     metadata will need to be cached somewhere (which can also include
>     being supplied to Teiid as an index file or DDL).  A possible
>     compromise is that VDBs in Teiid 8+ do have the ability to layer in
>     metadata, so it is possible to let the bulk of the metadata be
>     defined by "import foreign schema", but also to layer in additional
>     metadata as needed.  
> 
>     ----- Original Message -----
>     > I have a proposal on how to import translator based medadata in the
>     > Teiid Designer instead of creating a custom importer each time we add
>     > a
>     > new source.
>     >
>     > To support dynamic VDBs, Teiid team creates metadata facilities for
>     > most
>     > sources, it is prudent to use same facility in the Designer for
>     > importing metadata rather than creating another facility. Currently
>     > it
>     > is determined that "DatabaseMetadata" object is not sufficient and
>     > Teiid
>     > does not expose any Admin API method for this.
>     >
>     > In 8.0, Teiid defined DDL based matadata importer for a translator,
>     > if
>     > we write a reverse of this, where one can export DDL from the
>     > imported
>     > metadata then Teiid can provide a Admin API method for this, thus
>     > allowing the Teiid Designer to hook into Teiid and retrieve the
>     > metadata. Any thoughts?
>     >
>     > Thank you.
>     >
>     > Ramesh..
>     >
>     > _______________________________________________
>     > teiid-dev mailing list
>     > teiid-dev at lists.jboss.org
>     > https://lists.jboss.org/mailman/listinfo/teiid-dev
>     >
>     _______________________________________________
>     teiid-designer-dev mailing list
>     teiid-designer-dev at lists.jboss.org
>     https://lists.jboss.org/mailman/listinfo/teiid-designer-dev
> 
> 
> 
> 
> _______________________________________________
> teiid-dev mailing list
> teiid-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev