[teiid-dev] Connector supplied metadata

[John Doyle]

I've looked at the work checked in on the 7th, and given that it's the first check-in I think it's very good.  I have some technical comments that I will put in the JIRAs, but what's there is good.

My main point is about process.  We discussed the connector metadata feature before we were open source, and I thought the consensus was going the other way, it's hard to say from my remote perspective.  The next time I see it it's on the short list for 6.2.  But I read your responses below and I still don't know where this thing is going.  Is the plan to expose the "metadata objects" more broadly as THE way to specify metadata?

We need to hash out what a feature will be in the forums before we commit to it, so that everybody gets to see and pitch in if they choose, and some consensus can be built.  Everybody can see the implications so that people can react/plan.  It's not clear to me that the Designer team is signed up for any work associated with this.  I don't know if I'm going to try and adapt the SF connector to this new API because I don't know what's going to be there.  It looks like Extensions are not in the offing for this release?  Is that right, or wrong?

I think we're on the same page here technologically, I just don't have anyway to tell because the vision has not been elaborated.

~jd
----- "Ramesh Reddy" <rareddy at redhat.com> wrote:

> Please see in-line..
> 
> On Thu, 2009-07-09 at 13:56 -0400, John Doyle wrote:
> > 
> > Maybe the problem is that it has not been sufficiently explained,
> at
> > least not in a public forum.  
> 
> Based on my knowledge, we are not re-inventing the wheel here about
> metadata.Let me see if I can explain..(Steve correct if any
> mis-understandings on my part)
> 
> 1) Previously what we had was "teiid-metadata" project defined a
> runtime
> metadata object model, which was either generated from "eobjects" in
> the
> designer or from index files from the VDB either in designer or teiid
> runtime.
> 
> 2) During the 6.0 time, these metadata runtime objects were
> re-factored
> such that the notion of "eobjects" usage inside them was pushed into
> Designer code, but still left way to inject them and build these
> objects, to support preview and validation inside the Designer.
> 
> 3) Teiid run-time never directly accessed these runtime objects, they
> were always accessed through QMI (QueryMetadataInterface)
> 
> 4) Now during 6.2, these metadata runtime objects are further
> elevated
> into "connector-api" from "teiid-metadata" such that, a connector
> developer (on choice) can define the metadata of his/her connector
> dynamically using this metadata objects.
> 
> 5) so, in effect engine can now get the metadata from connectors 
> 
> connector --> metadata --> QMI --> Consumed by Engine
> designer --> eobjects --> metadata --> QMI --> Consumed by Engine
> (preview)
> designer --> index files(vdb) --> metadata --> QMI --> consumed by
> Engine (runtime)
> 
> > I don't have any problem with implementing a new vision over
> several
> > versions, which I suspect is what you're doing, but what's the
> > vision.  
> Yes, that is my understanding too. We may not able to fully convert
> all
> the connectors to support this notion of metadata due to UI
> requirements
> or  simply time/resource constrains.
> 
> > As far as I can tell from the public discussions and JIRAs, we are
> > implementation a new API to be used in connector scope, and have an
> > expectation that it will be grown into a full blown metadata API. 
> Yes and no. I know we have set a lower expectations in JIRA as to
> what
> at a minimum constitutes this feature. This not say to stop there.
> 
> >  That's backwards in my opinion.  We should be defining the best
> API
> > we can based upon the universe of potential environments/clients
> that
> > we envision for Teiid.  We can then implement over successive
> versions
> > in different components, right away in connectors if that's the
> > priority.
> > 
> If you think the current Designer based metadata model provides rich
> API
> to define/model a source, then this would be no different. I know
> this
> is not completely done yet, however I would encourage you take it for
> spin and identify the any short comings, so that we can capture them
> and
> fix them if time permitting. I agree that we need to put effort into
> putting out mature API and we are!
> 
> 
> Thanks
> 
> Ramesh..