Critical Designer VDB Artifact and VDB Editor changes
by Barry Lafond
John Verhaeg,
I'm getting bits of what you may be doing in the VDB framework from other posts, but I'm still not clear on the following issue:
>From a Designer standpoint... if we are going to be able to Deploy VDB's created in Designer, we need editable text fields for each "Source" model in the VDB Editor representing "Data Source", "Translator" and "JNDI" names.
Are you designing toward this requirement and can you give me an estimate on when you may have working code?
Barry LaFond
Teiid Designer Project Lead
16 years, 1 month
Re: [teiid-designer-dev] 7.0 Progress - 6/16/10
by Barry Lafond
2) Created Modeling > "Create Data Source" and "Delete Data Source" actions (maybe temporary actions) which extract Connection Profile from selected source model, creates JNDI name and creates a Teiid Data Source if it doesn't exist. Verifies the mechanics of storing connection info in models and using it create Teiid DS artifacts when necessary.
These should definitely be temporary. The VDB is responsible for creating the JNDI name and that name should certainly never be stored in the model since it's tied to the workspace UUID and path within that workspace.
I agree the actions should be temporary.
As far as JNDI name, since the JNDI name is really constructed with "Workspace" info (i.e. workspace uuid, project path, model name) AND DQP is responsible for communicating with Teiid (i.e. managing Data Sources) then the JNDI name should be managed in DQP. In all cases, the JNDI name should be given to the VDB when models are added, just like Translators... OR users given the option of "searching/selecting" existing Translators and Data Sources. (See my comments yesterday on https://jira.jboss.org/browse/TEIIDDES-411 )
3) On start-up, the ExecutionAdmin now has the smarts to search workspace for physical models and determine if models have matching JNDI names on the default Teiid server. This is a little tricky because we only get Data Source Names (JNDI) from Teiid API and can't get any properties to determine if the DS connection properties (i.e. type, URL, etc) match the Connection Profile info in a model. Based on our workspace logic to "clean-up" preview VDB artifacts on user events (close project, exit Designer, etc...) we are intending to NOT need to know anything more than the DS names.
We should always be able to match a model against its JNDI name without the need for properties. I don't understand why this would be tricky?
Simple scenario, 1) user creates relational model from one URL Oracle source. 2) User previews model (JNDI name is auto-generated) 3) User changes Connection Profile for Model to different URL Oracle Source 4) user previews model (JNDI name is auto-generated but DS already exists but with wrong URL. So we either need to "clean up JNDI" names when CP's change for model or find some other way to match. Without DS "type" or URL info, there's no way to know what's behind a DS on a Teiid Server.. short of opening up JON/JOPR. Again, JNDI names, though simple, are tied closely with Connection info, which is exclusive to the DQP. Hence DQP manages JNDI names as well as Data Sources on server. VDB's don't need to know anything but the value of the JNDI name string.
Barry
----- Original Message -----
From: "John Verhaeg" <jverhaeg(a)redhat.com>
To: "Barry Lafond" <blafond(a)redhat.com>
Cc: "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>
Sent: Wednesday, June 16, 2010 11:07:06 AM GMT -06:00 US/Canada Central
Subject: Re: [teiid-designer-dev] 7.0 Progress - 6/16/10
On Jun 16, 2010, at 10:34 AM, Barry Lafond wrote:
2) Created Modeling > "Create Data Source" and "Delete Data Source" actions (maybe temporary actions) which extract Connection Profile from selected source model, creates JNDI name and creates a Teiid Data Source if it doesn't exist. Verifies the mechanics of storing connection info in models and using it create Teiid DS artifacts when necessary.
These should definitely be temporary. The VDB is responsible for creating the JNDI name and that name should certainly never be stored in the model since it's tied to the workspace UUID and path within that workspace.
3) On start-up, the ExecutionAdmin now has the smarts to search workspace for physical models and determine if models have matching JNDI names on the default Teiid server. This is a little tricky because we only get Data Source Names (JNDI) from Teiid API and can't get any properties to determine if the DS connection properties (i.e. type, URL, etc) match the Connection Profile info in a model. Based on our workspace logic to "clean-up" preview VDB artifacts on user events (close project, exit Designer, etc...) we are intending to NOT need to know anything more than the DS names.
We should always be able to match a model against its JNDI name without the need for properties. I don't understand why this would be tricky?
4) Data Sources are now showing up in the Teiid server view, along with the available Translators. I'm currently providing a display name == model name instead of showing the actual generated JNDI because it'll be ugly and look something like: 8d9b22ab-f4ab-48c0-a6cd-b4d4501b81a5__TestProject_B.TestFolder.BooksSQL_B. User will see "BooksSQL_B"
Why are we showing translators in this view? What can the user do with that information? Seems like this is yet another topic we've been round and round on, coming to consensus every week, then rehashing it the following week...
The whole point of showing anything in this view was for manual user cleanup of stale, orphaned data sources due to application crashes, no-longer-used workspaces (this may currently be a moot point since we cleanup on application close), etc. The user needs to see the entire name to determine whether it's his artifact. Seems like this is yet another topic we've been round and round on, ... wow, deja-vu!
Thanks,
JPAV
16 years, 1 month
7.0 Progress - 6/16/10
by Barry Lafond
I haven't checked in code yet, but here's what I've got working locally.
1) Wired the Modeling > Deploy "VDB" action (and DND VDB from Explorer to Teiid server) to our ExectionAdmin. The deployment appears be successful to a point where there are Jboss "Translator" checks and it fails because the VDB's don't have translator info in them. Hoping that when VDB work is complete, this will just work.
2) Created Modeling > "Create Data Source" and "Delete Data Source" actions (maybe temporary actions) which extract Connection Profile from selected source model, creates JNDI name and creates a Teiid Data Source if it doesn't exist. Verifies the mechanics of storing connection info in models and using it create Teiid DS artifacts when necessary.
3) On start-up, the ExecutionAdmin now has the smarts to search workspace for physical models and determine if models have matching JNDI names on the default Teiid server. This is a little tricky because we only get Data Source Names (JNDI) from Teiid API and can't get any properties to determine if the DS connection properties (i.e. type, URL, etc) match the Connection Profile info in a model. Based on our workspace logic to "clean-up" preview VDB artifacts on user events (close project, exit Designer, etc...) we are intending to NOT need to know anything more than the DS names.
4) Data Sources are now showing up in the Teiid server view, along with the available Translators. I'm currently providing a display name == model name instead of showing the actual generated JNDI because it'll be ugly and look something like: 8d9b22ab-f4ab-48c0-a6cd-b4d4501b81a5__TestProject_B.TestFolder.BooksSQL_B. User will see "BooksSQL_B"
Rest of team:
John Doyle is working on providing hooks to Preview Data using DTP Transient connection profiles and Eclipse's "Run" action paradigm. Result will be Select Relational Table, right-click "Preview Data". If info missing "i.e. need to select Connection Profile" or Translator, then user is query'd. Then results view is opened to view .... the results.
Dan Florian is working on the Teiid artifact management framework to make the Preview Data work. Since querying data, is really executing queries on a deployed VDB, Designer is utilizing the "merge VDB" capabilities to deploy atomic VDBs (one model per VDB) and use the merge logic for "dependencies". There is also a fair amount of logic required to coordinate these steps and keep the Teiid deployed artifacts in sync with basic Eclipse workspace operations (Open/Close Project, Delete/Move Model, Save Model, etc..). In addition there are additional process/steps to perform when user changes Default Teiid server.
John Verhaeg is working on the necessary changes/fixes to the VDB model and VDB Editor framework to be compatible with Teiid API, VDB schema.
Barry
16 years, 1 month
Re: [teiid-designer-dev] Critical Designer VDB Artifact and VDB Editor changes
by Barry Lafond
Bottom line is, I really have no idea what you are doing or when you plan to be done with any part of it.
I don't need all the details. I'm only asking for enough info that I can judge when we'll be ready for any given milestone based on input from all project team members. That includes you.
Unfortunately, "working on it now", doesn't help me at all. Could you elaborate on what "it" is and give me an estimate on when you plan to have it done?
Barry
----- Original Message -----
From: "John Verhaeg" <jverhaeg(a)redhat.com>
To: "Barry Lafond" <blafond(a)redhat.com>
Cc: "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>
Sent: Monday, June 14, 2010 3:30:12 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-designer-dev] Critical Designer VDB Artifact and VDB Editor changes
On Jun 14, 2010, at 2:17 PM, Barry Lafond wrote:
How can we make this happen?
Working on it now. Not sure what you're really asking for here.
Thanks,
JPAV
16 years, 1 month
Critical Designer VDB Artifact and VDB Editor changes
by Barry Lafond
John,
As of today's code-base, the VDB editor framework is a step-behind the schema defined by Teiid. Namely, there is no placeholder or accommodation for an attribute in the "source" element to handle a "translator-name". Without this info, the VDB's aren't consumable by Teiid.
Also, as of today's code-base, I can't create a new VDB in Designer due to the following... (this worked a week ago)
java.lang.NoClassDefFoundError: org/teiid/designer/vdb/Vdb
at com.metamatrix.modeler.internal.vdb.ui.editor.VdbEditor.init(VdbEditor.java:544)
at org.eclipse.ui.internal.EditorManager.createSite(EditorManager.java:798)
Per our weekly conference call today, providing the Teiid Tooling necessary to at least create a valid Teiid VDB artifact seems to be critical to producing a demo-able 7.0 Release sometime in the next week to 10 days.
How can we make this happen?
Barry LaFond
Teiid Designer Project Lead
16 years, 1 month
7.0 Designer progress
by Barry Lafond
FYI, we're running on all cylinders, now that Teiid Admin API has been flushed out for the "last" time :) These changes do, however, have a rippling affect in both our new VDB editing framework and our connection framework, so it'll take a bit to get back to creating deployable VDBs. Don't have an estimate yet, but I'll keep you'all posted.
We're also working on the both the underlying framework and UI components for restoring the Preview Data functionality.
Lastly, to help get 7.0 out the door sooner than later we're pushing the following tasks to 7.1 release.
1) Data Role management
• This is feature that has never existed in Designer but is intended to replace the MMX console functionality, so it is critical for 7.1 release.
2) Translator properties override
• The connection management framework targeted for 7.0 involves referencing "translator-name"s in VDB vdb.xml file. The Teiid API provides the hooks to select from a list of existing default Translator names, but does not allow creating new Translators. They have provided a mechanism to "override" the default properties via additional XML entries in the vdb.xml file. It's not critical to add this functionality for 7.0, so it'll be pushed to 7.1 as well.
Barry LaFond
Teiid Designer Project Lead
16 years, 1 month
Re: [teiid-designer-dev] Teiid Designer Connections paradigm
by John Doyle
----- "John Verhaeg" <john.verhaeg(a)gmail.com> wrote:
>
On Jun 4, 2010, at 11:46 AM, John Doyle wrote:
>
But do we know yet how the connection profile properties and their values will map to whatever properties Teiid understands?
>
> JD-For each 'type' of profile (JDBC, SF, WSDL) there will be a fixed set of properties. So we will have to create a fixed mapping for each Profile-Trans/DS pairing. Currently the only profiles I've got are JDBC, so we can start with those, and then follow on with SF when I create it.
>
>
So that means the use of custom connectors from Designer will also require the development of new DTP connection profile, an extension to this mapping, etc.?
Yes, assuming if the custom connector developer wants to go that far.
>
Thanks,
>
JPAV
>
>
16 years, 1 month
Teiid Designer Connections paradigm
by Barry Lafond
Wanted to update you'all on where our current connection management will look like for 7.0 and some background as to why we got here, so please bare with me.
General Info:
1) Designer is providing access to the Teiid Admin API methods necessary to deploy or manage the artifacts necessary to deploy a VDB and execute. These artifacts include: Data Source (connection factory xml fragment) and the VDB itself. The VDB must contain a vdb.xml configuration file containing the source model connection information including a “ translator-name ” and the “ connection-jndi-name ” (or data source name).
2) DTP provides Teiid Designer the framework for creating and managing “Data Source” objects, known as Connection Profiles. These Connection Profiles contain the basic URL & User-specific properties used to access database schema during Designer's JDBC Import process and provide the properties necessary for Teiid to create it's JDBC connection for data access during VDB execution. For 7.0, we'll have to provide Connection Profile contributions for SF, Text & LDAP in order to allow users to construct valid VDB's containing these model types.
3) Teiid Admin functionality is provided by the web-based JOPR/JON tool. This tool enables creation and management of the connection artifacts specific to your data sources, in particular, Connection Factories. Teiid Designer utilizes Teiid's Admin API to hand-off the pertinent Connection Profile information for each source, when necessary, to create the appropriate connection artifacts for the user.
Replacing JdbcSource Info with Connection Profile Info
To facilitate transportability of both models and VDB's, we're proposing injecting a more generic set of connection properties into a model via a coded “Annotation” object containing Connection Profile info. This allows Designer to persist/retrieve connection info for both jdbc and non-jdbc sources and it eliminates the need to manage “source – connection” bindings within the workspace (ala MMX & 6.0-6.3).
This means the JDBC Source properties currently stored in imported Relational models, will be deprecated. Any legacy model (pre-7.0) that is loaded into new Designer workspace would be required to have it's connection profile set prior to Previewing.
This provides a clean scenario for Relational models created via ANY manner. When user wishes to Preview or Execute, then the user will be required to either select existing, or create a new “Connection Profile" so we can sync up/populate the model with required connection info.
So if user A creates a ZZZ.vdb and checks it into SVN, then user B checks out ZZZ.vdb into their workspace without checking out the models, then we can still extract the necessary info to create the necessary Connection Profile in their workspace.
Basically this condenses the "Connection" management in Designer to always leverage DTP's existing connection profile framework.
Barry
16 years, 1 month
