[teiid-dev] Connector Based Metadata

[Ken Johnson]

Ramesh Reddy wrote:
> On Wed, 2009-06-17 at 11:47 -0400, John Doyle wrote:
>   
>> If I understand the proposal, the connector API would be expanded so
>> the connector can provide the metadata of the source system via an
>> object representation akin to JDBC metadata.  This would allow a user
>> of Teiid embedded to create bindings to source systems and query those
>> systems without any model or VDB.  Users would be able to submit
>> queries to Teiid that joined across the source systems much like they
>> would write in a transformation for a view model.  Is all of this
>> correct?
>>
>>     
> Yes, that is my understand as well.
>
>   
>> Unless I'm missing some additional utility of this approach, I don't
>> really see much value in this approach.  On the contrary, I think it
>> undermines the value of Teiid by offering federation without much
>> abstraction.  I don't see how this leads to application code that
>> significantly less brittle than what users would write without us, and
>> I think it leads users down a dead end. Much of the code they create
>> around this approach would likely be thrown away were they to adopt
>> full usage of Teiid.
>>
>>     
> Can you explain why you think there is additional utility? Are you
> thinking for example taking text connector, we need a utility to convert
> the def file into a metadata format? or like DDL for JDBC connector to
> metadata format?
>
> What Teiid does is Federation at its core, abstraction is layer above
> that which provides us to define much more manageable schema independent
> of sources. You are correct on that respect. But, I would disagree, that
> there is no value. The code is no more brittle than what they would have
> written themselves, however the they would not have federation. That is
> what we fill in, with very easy processing model.
>   
True that the code written in the usage model is no more brittle than if 
the user was to code directly to a database.  However it is more brittle 
(at least from the application perspective) than if they wrote to an 
abstract view.   So this is valuable to a developer that's comfortable 
with tight coupling but lacks federation functionality.  This capability 
does not directly provide value to the user interested in data source 
virtualization and loose coupling.  That's fine as long as the project 
is clear that these modes of use are distinct and each have their 
positives/negatives. 
> Whether they throw away or not, my hunch is this will satisfy the needs
> of majority of the developers, and it is quick way somebody can see the
> benefits of Teiid based data integration.  
>
>   
It's a quick way to demonstrate/achieve *federation* in it's simplest 
form.  However Teiid provides substantial value above federation.  That 
should not get lost nor compromised by this effort.
>> I also think that by doing this now, we are potentially missing an
>> opportunity to improve our integration story for community members who
>> might want to integrate their products with Teiid.  There has been a
>> longstanding desire to reduce the effort required to create an
>> importer, and there is also a requirement for alternative tooling.
>> There are a lot of balls up in the air right now and I think we should
>> take this opportunity to look at the all of APIs that we are going to
>> expose and rationalize them as much as possible.
>>
>>     
>
> This is not about tooling, this about alternative ways to defining the
> metadata to be used by Teiid, independent of any tooling.
>   
>   
>> My preferred approach would be to answer the big question first.  What
>> is our canonical representation of metadata?  Is it DDL, or something
>> else?  Once we've done that, then we should define the APIs and tools
>> to create the metadata, and the hooks to provide it. 
>>     
> These two are separate concerns, granted they will compliment each
> other. We are committed for defining this, there is already discussion
> happening in the community as to how we want to approach this.
>
> What we want to do now is define the runtime model we need for the
> engine, then whatever the persistent form may that be DDL or SQL/MED or
> XML will defined in coming releases, that produces this runtime model.
>   
I'm a bit unclear here.  There's a need to define the runtime model we 
need for the engine?
> _______________________________________________
> teiid-dev mailing list
> teiid-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
>   


-- 
Ken Johnson
Sr. Product Manager
Data Services/MetaMatrix
JBoss, a division of Red Hat
978.392.3917
ken.johnson at redhat.com