[teiid-dev] Connector Based Metadata
On Wed, 2009-06-17 at 11:47 -0400, John Doyle wrote:
If I understand the proposal, the connector API would be expanded so
the connector can provide the metadata of the source system via an
object representation akin to JDBC metadata. This would allow a user
of Teiid embedded to create bindings to source systems and query those
systems without any model or VDB. Users would be able to submit
queries to Teiid that joined across the source systems much like they
would write in a transformation for a view model. Is all of this
correct?
Yes, that is my understand as well.
Unless I'm missing some additional utility of this approach, I
don't
really see much value in this approach. On the contrary, I think it
undermines the value of Teiid by offering federation without much
abstraction. I don't see how this leads to application code that
significantly less brittle than what users would write without us, and
I think it leads users down a dead end. Much of the code they create
around this approach would likely be thrown away were they to adopt
full usage of Teiid.
Can you explain why you think there is additional utility? Are you
thinking for example taking text connector, we need a utility to convert
the def file into a metadata format? or like DDL for JDBC connector to
metadata format?
What Teiid does is Federation at its core, abstraction is layer above
that which provides us to define much more manageable schema independent
of sources. You are correct on that respect. But, I would disagree, that
there is no value. The code is no more brittle than what they would have
written themselves, however the they would not have federation. That is
what we fill in, with very easy processing model.
Whether they throw away or not, my hunch is this will satisfy the needs
of majority of the developers, and it is quick way somebody can see the
benefits of Teiid based data integration.
I also think that by doing this now, we are potentially missing an
opportunity to improve our integration story for community members who
might want to integrate their products with Teiid. There has been a
longstanding desire to reduce the effort required to create an
importer, and there is also a requirement for alternative tooling.
There are a lot of balls up in the air right now and I think we should
take this opportunity to look at the all of APIs that we are going to
expose and rationalize them as much as possible.
This is not about tooling, this about alternative ways to defining the
metadata to be used by Teiid, independent of any tooling.
My preferred approach would be to answer the big question first.
What
is our canonical representation of metadata? Is it DDL, or something
else? Once we've done that, then we should define the APIs and tools
to create the metadata, and the hooks to provide it.
These two are separate
concerns, granted they will compliment each
other. We are committed for defining this, there is already discussion
happening in the community as to how we want to approach this.
What we want to do now is define the runtime model we need for the
engine, then whatever the persistent form may that be DDL or SQL/MED or
XML will defined in coming releases, that produces this runtime model.