Re: [teiid-designer-dev] [teiid-dev] VDB Versioning Feature
by Steven Hawkins
Before responding to you're conclusions below, I'd like to bring up that we currently report the vdb name as the catalog name through our system tables / jdbc metadata. So to remove vdb version we would either need to:
- not report the vdb name as the catalog to prevent apparent behavioral differences
- or change the vdb name in the vdb.xml to mean the catalog name
I assume we would take the latter approach. In that world creating a copy of a vdb does not require modifying the vdb.xml, just creating a new file. However we still need a unique way to refer to each vdb, so I would propose using the filename (designer already starts off with the assumption filename = vdbname). While it's possible through direct file deployments to do all kinds of crazy things (vdbs inside of zip files, sub folders, etc.), we can assume we're really only interested in this discussion with our "managed" view of the deploy directory. In which case we effectively do control the .vdb and -vdb.xml artifacts there.
To your points:
a) We could forgo even adding a copy vdb method if we put versioning as a primary concern in the deployVDB admin method. There are a couple of possible variants:
- overwrite the previous
- give a different filename to the vdb to deploy (pseudo active default behavior)
- give a different filename to the old vdb (which would be nothing more than a combined copy then deploy operation)
The current admin api method for vdb deployment can handle the first two cases - void deployVDB(InputStream contents, String fileName). It would probably make more sense in the last case for the admin to specify the name rather than to use auto-increment logic since a proper name allows you to know in advance how to change your urls. This means that we only have to add 1 additional method - boolean deployVDB(InputStream contents, String fileName, String oldVdbName) returning true if the old file existed. The associated tooling would then need to have deployment screens that specify if this deploy is an overwrite or what the backup name should be.
b) Other than keeping the vdb name / file name the same between deployments, there wouldn't be deployment related versioning concerns at design-time.
c) Seems like this follows from a.
----- Original Message -----
From: "Ken Johnson" <kejohnso(a)redhat.com>
To: "Steven Hawkins" <shawkins(a)redhat.com>
Cc: "Larry O'Leary" <loleary(a)redhat.com>, "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>, "teiid-dev" <teiid-dev(a)lists.jboss.org>
Sent: Wednesday, April 21, 2010 12:49:50 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] [teiid-designer-dev] VDB Versioning Feature
I think this one has been flogged pretty well but I'd like to take
another crack at defining an alternative to the current version
number-based approach.
The principle value of the exising VDB versioning is as Ramesh
summarized: Client Transparency. We want a way for administrators to
introduce new versions of VDBs and seamlessly migrate clients that are
prepared to migrate as well as provide a simple means to allow laggard
clients to continue using an older/deprecated version of the VDB.
It seems this can be achieved largely through naming convention and good
process, with support from APIs/tools. From a user experience point of
view, the old solution had numerous benefits:
1. auto-version-increment prevented accidental overwrites on deployment
and eliminated the need for convention or best-practice
2. the name+ version number approach made it clear to administrators
that a set of VDBs are really the same thing but at different revision
levels.
3. the active-default allowed administrators to control the vdb used by
clients without disrupting the client code/configuration and didn't
break active client connections.
The replacement solution, though different in implementation, should
strive to have a similar level of user-friendliness:
Deployment via AdminAPI/shell and admin-console provide an opportunity
to constructively guide the user. For example warning them when
deploying a VDB with the same name as a currently deployed VDB. Or
giving them an option to rename the existing VDB to a "backup" name
prior to deploying the new version. We should help the user to not shoot
themselves in the foot. Deployment via file drop/copy doesn't afford
much opportunity to help users.
Best practices/scenarios should be documented so users understand how to
manage VDB "versions", not just in production but across the various
lifecycle stages (dev->test->prod). One usage scenario is for ModeShape
to be used as a repository of artifacts like VDB where administrators
can locate the "approved-for-deployment" VDB and then deploy it via URI
using AdminAPI/shell or admin-console. The best practices should provide
guidance here as well.
So, IMO option 1 can work if:
a) API and tooling operations are added to assist/protect the user
b) the Design-time developer is not responsible for anything related to
version management. This is an administrator activity.
c) most importantly: This continues to be offered as a feature. By that
I mean, this user need is acknowledged and Teiid provides
facilities/docs that address "VDB version management" as a capability
(even if multi-step) and not just "something a user can do if so inclined"
Thoughts?
-Ken
Steven Hawkins wrote:
> Is there any consensus here? Mike, Ken?
>
> As I see it:
>
> Option 1: get rid of the version concept altogether and possibly add a rename or copy vdb operation.
>
> Option 2: retain the version concept along with the active default designation. However setting the version will be a design time responsibility and file drop deployments will require different vdb file names for different versions.
>
> Option 3: ?
>
> And regardless of the scenario, deployers will need to ensure that only truly active vdbs are in an active state or suffer the memory/load costs.
>
> ----- Original Message -----
> From: "Larry O'Leary" <loleary(a)redhat.com>
> To: "Steven Hawkins" <shawkins(a)redhat.com>
> Cc: "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>, "teiid-dev" <teiid-dev(a)lists.jboss.org>, "John Verhaeg" <jverhaeg(a)redhat.com>
> Sent: Thursday, February 18, 2010 4:13:03 PM GMT -06:00 US/Canada Central
> Subject: Re: [teiid-designer-dev] [teiid-dev] VDB Versioning Feature
>
> On Thu, 2010-02-18 at 15:43 -0500, Steven Hawkins wrote:
>
>> At first glance in the scenario you list below, there's no need to
>> even bother with the promotion concept. You would just install the
>> new version (with a new name) and let apps point over to it as they
>> are ready.
>>
>
> Well, except for the fact that the clients do not want to change their
> connection properties. In other words, whatever the business as a whole
> deems as the latest and greatest, they are fine with. It is just that
> there is normally some stragglers that do things their way and disregard
> the business decision. In these cases, these stragglers would be
> responsible for updating their client connection properties to point to
> the renamed (previous version) VDB. Must like in legacy, they would
> update their connection properties to point to the old version.
>
>
>> To achieve the flow your suggesting without versioning would actually
>> require an extra step. Prior to promoting the next version, you would
>> need to deploy a duplicate of the current version with a new name.
>> Then the small set of apps not ready for the update would need to have
>> their connections pointed to the copy. Then you could safely promote
>> the next version without interrupting the lagging apps.
>>
>
> Right. In this scenario the new VDB would get the name of the old VDB
> so that client apps that were following the business decision would not
> need to make a change to get the newer VDB. And the lagging users would
> update their clients to point to the renamed VDB.
>
> I am not familiar with how we are packaging and naming VDBs to achieve
> this but in my opinion, making this a manual process seems logical and
> there would be no need for the system to automatically keep track of
> things.
>
> Furthermore, the functionality of "active-default" can be achieved the
> same way. Just make the active-default or latest version a base name.
> MyCoolSource.vdb. When a new version is available for integration
> testing but you are not ready to promote it to the "active-default" or
> "latest-version", deploy it as MyCoolSource_NextGen.vdb. In this case,
> users who want to use the NextGen version would simply change their
> client app to point to MyCoolSource_NextGen.vdb and everyone else will
> continue to use the "active-default" of MyCoolSource.vdb.
>
> When you are ready to move the "active-default" to the contents of
> MyCoolSource_NextGen.vdb then you simply copy MyCoolSource.vdb to
> MyCoolSource_OldGen.vdb, copy MyCoolSource_NextGen.vdb to
> MyCoolSource.vdb and voila. The only downside here is that
> MyCoolSource.vdb and MyCoolSource_NextGen.vdb are duplicates but I
> really don't see this as a problem.
>
> The naming convention would be up to business rules and we might want to
> make some kind of best practice suggestions to make this manageable but
> concept could be just as easily accomplished with version numbers in the
> name:
>
> MyCoolSource_v1.vdb
> MyCoolSource_v2.vdb
> MyCoolSource_v3.vdb
>
> And to achieve "active-default" or "latest-version" you would use
> MyCoolSource.vdb -> MyCoolSource_v3.vdb as a symbolic link or copy of
> the version you wanted to use.
>
> Of course, this logic all falls apart if the VDB actually contains a
> descriptor that defines its name to the system in which case the process
> is the same but a bit more complicated because you wouldn't be able to
> simply copy/symlink.
>
>
>
>> ----- Original Message -----
>> From: "Larry O'Leary" <loleary(a)redhat.com>
>> To: "John Verhaeg" <jverhaeg(a)redhat.com>
>> Cc: "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>, "teiid-dev" <teiid-dev(a)lists.jboss.org>
>> Sent: Wednesday, February 17, 2010 11:38:00 AM GMT -06:00 US/Canada Central
>> Subject: Re: [teiid-designer-dev] [teiid-dev] VDB Versioning Feature
>>
>> On Wed, 2010-02-17 at 11:27 -0600, John Verhaeg wrote:
>>
>>> On Feb 17, 2010, at 10:44 AM, Larry O'Leary wrote:
>>>
>>>
>>>> Well there are two possibilities here. One is that there is a
>>>> roll-back. The other is that a new version of a VDB has been chosen as
>>>> the default but some of the organization is not ready to use the newer
>>>> version. In which case they would modify their clients to use the
>>>> previous version until they are ready to use the new one.
>>>>
>>>> So, in the "roll-back" scenario, the change is transparent to the
>>>> clients while in the "not-ready-to-upgrade" scenario the administrative
>>>> change is placed on the client. These are the two cases we have with
>>>> legacy customers.
>>>>
>>> Maybe I'm misunderstanding, but I don't believe your "new version"
>>> scenario makes practical sense. If an organization is implementing a
>>> new version and not ready to force that change on certain
>>> applications, departments, etc., wouldn't they want just that - no
>>> change to those areas? They're likely not going to impose the need to
>>> change a slew of existing clients to use the old VDB just to enable
>>> new clients to use the previous VDB name. Rather, they'd want to
>>> deploy the new VDB with a new name that only the new clients use.
>>> Then migration to the new VDB name for other clients could be easily
>>> scheduled and managed. Seems like the alternative would be a
>>> change-control nightmare.
>>>
>>>
>> Here is how it works. All clients are using an existing VDB and
>> development creates a new version of that VDB. They put it through all
>> the testing and identify any possible conflicts and changes that a
>> client may need to make. The VDB is then slated to be migrated to
>> production during their quarterly maintenance window. However, a small
>> group within the organization have not been able to complete all of
>> their testing necessary to use the new version of the VDB. In this
>> case, a business decision is made to proceed with the migration into
>> production and the "small group" is notified that they have until MM/dd
>> to complete their testing or they will need to point their clients at an
>> old version of the VDB. The old version will be left in production but
>> as a deprecated source.
>> --
>>
>> Larry O'Leary
>> Middleware Support Engineering Group
>> Global Support Services
>> Red Hat, Inc.
>> loleary(a)redhat.com
>> 1.866.LINUX70 (+1.314.336.2990) xt 81-62909
>> Office: +1.314.336.2909
>> Mobile: +1.618.420.7623
>> --
>> Looking to carve out IT costs?
>> http://www.redhat.com/carveoutcosts/
>>
>> _______________________________________________
>> teiid-designer-dev mailing list
>> teiid-designer-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/teiid-designer-dev
>>
>
>
14 years
Re: [teiid-designer-dev] [teiid-dev] VDB Versioning Feature
by Steven Hawkins
Is there any consensus here? Mike, Ken?
As I see it:
Option 1: get rid of the version concept altogether and possibly add a rename or copy vdb operation.
Option 2: retain the version concept along with the active default designation. However setting the version will be a design time responsibility and file drop deployments will require different vdb file names for different versions.
Option 3: ?
And regardless of the scenario, deployers will need to ensure that only truly active vdbs are in an active state or suffer the memory/load costs.
----- Original Message -----
From: "Larry O'Leary" <loleary(a)redhat.com>
To: "Steven Hawkins" <shawkins(a)redhat.com>
Cc: "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>, "teiid-dev" <teiid-dev(a)lists.jboss.org>, "John Verhaeg" <jverhaeg(a)redhat.com>
Sent: Thursday, February 18, 2010 4:13:03 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-designer-dev] [teiid-dev] VDB Versioning Feature
On Thu, 2010-02-18 at 15:43 -0500, Steven Hawkins wrote:
> At first glance in the scenario you list below, there's no need to
> even bother with the promotion concept. You would just install the
> new version (with a new name) and let apps point over to it as they
> are ready.
Well, except for the fact that the clients do not want to change their
connection properties. In other words, whatever the business as a whole
deems as the latest and greatest, they are fine with. It is just that
there is normally some stragglers that do things their way and disregard
the business decision. In these cases, these stragglers would be
responsible for updating their client connection properties to point to
the renamed (previous version) VDB. Must like in legacy, they would
update their connection properties to point to the old version.
> To achieve the flow your suggesting without versioning would actually
> require an extra step. Prior to promoting the next version, you would
> need to deploy a duplicate of the current version with a new name.
> Then the small set of apps not ready for the update would need to have
> their connections pointed to the copy. Then you could safely promote
> the next version without interrupting the lagging apps.
Right. In this scenario the new VDB would get the name of the old VDB
so that client apps that were following the business decision would not
need to make a change to get the newer VDB. And the lagging users would
update their clients to point to the renamed VDB.
I am not familiar with how we are packaging and naming VDBs to achieve
this but in my opinion, making this a manual process seems logical and
there would be no need for the system to automatically keep track of
things.
Furthermore, the functionality of "active-default" can be achieved the
same way. Just make the active-default or latest version a base name.
MyCoolSource.vdb. When a new version is available for integration
testing but you are not ready to promote it to the "active-default" or
"latest-version", deploy it as MyCoolSource_NextGen.vdb. In this case,
users who want to use the NextGen version would simply change their
client app to point to MyCoolSource_NextGen.vdb and everyone else will
continue to use the "active-default" of MyCoolSource.vdb.
When you are ready to move the "active-default" to the contents of
MyCoolSource_NextGen.vdb then you simply copy MyCoolSource.vdb to
MyCoolSource_OldGen.vdb, copy MyCoolSource_NextGen.vdb to
MyCoolSource.vdb and voila. The only downside here is that
MyCoolSource.vdb and MyCoolSource_NextGen.vdb are duplicates but I
really don't see this as a problem.
The naming convention would be up to business rules and we might want to
make some kind of best practice suggestions to make this manageable but
concept could be just as easily accomplished with version numbers in the
name:
MyCoolSource_v1.vdb
MyCoolSource_v2.vdb
MyCoolSource_v3.vdb
And to achieve "active-default" or "latest-version" you would use
MyCoolSource.vdb -> MyCoolSource_v3.vdb as a symbolic link or copy of
the version you wanted to use.
Of course, this logic all falls apart if the VDB actually contains a
descriptor that defines its name to the system in which case the process
is the same but a bit more complicated because you wouldn't be able to
simply copy/symlink.
> ----- Original Message -----
> From: "Larry O'Leary" <loleary(a)redhat.com>
> To: "John Verhaeg" <jverhaeg(a)redhat.com>
> Cc: "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>, "teiid-dev" <teiid-dev(a)lists.jboss.org>
> Sent: Wednesday, February 17, 2010 11:38:00 AM GMT -06:00 US/Canada Central
> Subject: Re: [teiid-designer-dev] [teiid-dev] VDB Versioning Feature
>
> On Wed, 2010-02-17 at 11:27 -0600, John Verhaeg wrote:
> > On Feb 17, 2010, at 10:44 AM, Larry O'Leary wrote:
> >
> > > Well there are two possibilities here. One is that there is a
> > > roll-back. The other is that a new version of a VDB has been chosen as
> > > the default but some of the organization is not ready to use the newer
> > > version. In which case they would modify their clients to use the
> > > previous version until they are ready to use the new one.
> > >
> > > So, in the "roll-back" scenario, the change is transparent to the
> > > clients while in the "not-ready-to-upgrade" scenario the administrative
> > > change is placed on the client. These are the two cases we have with
> > > legacy customers.
> >
> > Maybe I'm misunderstanding, but I don't believe your "new version"
> > scenario makes practical sense. If an organization is implementing a
> > new version and not ready to force that change on certain
> > applications, departments, etc., wouldn't they want just that - no
> > change to those areas? They're likely not going to impose the need to
> > change a slew of existing clients to use the old VDB just to enable
> > new clients to use the previous VDB name. Rather, they'd want to
> > deploy the new VDB with a new name that only the new clients use.
> > Then migration to the new VDB name for other clients could be easily
> > scheduled and managed. Seems like the alternative would be a
> > change-control nightmare.
> >
>
> Here is how it works. All clients are using an existing VDB and
> development creates a new version of that VDB. They put it through all
> the testing and identify any possible conflicts and changes that a
> client may need to make. The VDB is then slated to be migrated to
> production during their quarterly maintenance window. However, a small
> group within the organization have not been able to complete all of
> their testing necessary to use the new version of the VDB. In this
> case, a business decision is made to proceed with the migration into
> production and the "small group" is notified that they have until MM/dd
> to complete their testing or they will need to point their clients at an
> old version of the VDB. The old version will be left in production but
> as a deprecated source.
> --
>
> Larry O'Leary
> Middleware Support Engineering Group
> Global Support Services
> Red Hat, Inc.
> loleary(a)redhat.com
> 1.866.LINUX70 (+1.314.336.2990) xt 81-62909
> Office: +1.314.336.2909
> Mobile: +1.618.420.7623
> --
> Looking to carve out IT costs?
> http://www.redhat.com/carveoutcosts/
>
> _______________________________________________
> teiid-designer-dev mailing list
> teiid-designer-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-designer-dev
--
Larry O'Leary <loleary(a)redhat.com>
Red Hat, Inc.
14 years
Re: [teiid-designer-dev] Teiid Designer 7.0 Release Planning
by John Doyle
----- "Barry Lafond" <blafond(a)redhat.com> wrote:
>
> The following is a culmination of various discussions over the past month around what Teiid Designer 7.0 should look like or how it should be behave based on fundamental changes to the Teiid server. These changes mainly affect Admin API and migration to using the JCA compliant container technology compatible with JBoss AS.
>
> These changes have required us to re-architect Designer's connector management framework as well as a rewrite the VDB artifact creation, editing and management framework.
>
> Note that the new target "platform" for Designer includes integration with JBoss Tools plus the availability of JOPR for Teiid Server management.
>
> Please view these plans and all feeback and suggestions are welcome.
>
> Barry LaFond
> Teiid Designer Project Lead
>
> ===============================================
> Basic Functionality
>
A) Designer is a Teiid design tool required for creating consumable/deployable VDB's containing indexed/queryable models.
>
> B) Designer is NOT a Connector/Connection Factory (CF) management tool. (JOPR/JON is)
>
> - Note that in Designer 6.0 thru 6.3 Connectors were refered to as "Connector Types" and Connection Factories were called "Connector Bindings". Designer 7.0 will be adapting the new terminology.
>
> C) Designer will provide a Teiid View (name TBD) which exposes viewable/filterable content of one or more "Teiid Servers". (Similar functionality to Release 6.3's Connectors View)
>
> - Designer allows connecting to and viewing contents of multiple Teiid Servers
> - Shows Connectors as children of Server
> - Shows CF's as children of Connectors
> - Shows "Source Models" (bindings) as children to CF's
> - Bind/Unbind features remain similar to 6.3 release
> - NO CF info is persisted within Designer. All of it is persisted and accessed through Teiid.
> - View will provide "filters" to hide Connectors if so desired. (current 6.3 feature)
> - View will provide "pivot" feature to show "SourceBinding-centric" view (Stretch goal)
> - View will support the notion of a "Default Teiid Server". Preview Data and VDB development will be tied to the current "Default Teiid Server".
> - Users can change their "Default Teiid Server" at any time
>
>
• Not sure how Designer reacts to switching "Default Teiid Server"
• Could allow migration of CF's from old Default to New Default
• ... or assume user will take care of it? >
If we go to the length of migrating CF's, then I think we should make it configurable. For instance, when the user changes the default, we pop up a dialog asking if they want to migrate the CFs to the new default "in order to enable execution of the current model projects on the new default". We could limit it to open model projects, and add a checkbox on the dialog to "remember my choice".
What about Connectors? If we switch defaults and the new default is lacking a connector? I don't think Designer should migrate Connectors, but we should fail gracefully with an appropriate message and migrate projects that won't work in the new default.
> D) Designer will assist when needed to help user create/select CF's to either Preview Data or construct deployable VDB's.
>
>
> - Any Relational model created via JDBC import that contains JDBC Import info will be a candidate for auto-discovery of existing CF.
> - All other Relational models will required user-intervention to select an existing CF or create a compatible CF from existing Connector (i.e. type)
> - CF's are named with respect to the corresponding bound "Model name"
> - CF's created from Designer are managed from Designer and are "Model-centric". If a model is deleted from Designer, the corresponding CF is deleted from server.
> - Users can "Edit" a CF from Designer to change basic properties, reset password, etc...
> - Users cannot Create CF from Designer unless directed to via "Preview Data" or VDB editing.
>
So when does a user create the ConnectionFactory? During VDB creation (OK get that), or after they hit the "Running Man" button? Historically that button was deactivated until the model was bound, so is the action now that the button is always active and when the user clicks it for the first time for a model they are prompted to create the CF?
I just want to add in here the wrinkel that XML-Relational models should never be previewed, so it would be nice if I could deactivate preview all together for certain types of models.
> E) Designer will NOT perform auto-CF creation or matching during JDBC import (ala MM 5.5.3 & pre Teiid Designer 7.0)
>
> - The reasoning behind this change is a result of the role that Designer is now targeted to fill within the JBoss Tools enviroment, which is to create and test artifacts for deployment to Teiid Servers.
> - Instead of assuming that a users will always want to "Preview Data", for instance, we are now assuming that the may want to, so we are delaying the CF management part until actually needed.
>
> F) Designer will implement a "Hidden VDB" framework to allow Preview Data functionality.
>
> - Preview Data functionality was provided "Pre-7.0 Designer" via separate embedded Teiid code. Maintenance of this code has been expensive, so to speak, so removing that requirement has been a goal of Teiid & Teiid Designer.
> - The simplicity and flexibility of Teiid's ability to deploy VDB's has provided a mechanism to replace that embedded Teiid source code with a deployed "Hidden VDB" which is virtually 100% compatible with the VDB execution feature which is already a Designer requirement.
> - Bindings of "Source" models to CF's will be displayed in Designer's Connectors View (Name of view TBD... maybe Teiid? Execution?)
> - This hidden vdb will not be exposed to the user.
> - Hidden VDB will be created/deployed when "Preview Data" is initially requested.
> - Hidden VDB will be "Teiid-Server Centric" (i.e. one per server)
>
>
Will this be a single hidden VDB for a designer connection to Teiid, or a hidden VDB per model project? I ask because I'm concerned about name collisions that might requires us to use fully qualified SQL in the preview queries.
===============================================
> Related issues
>
ISSUE 1: Opening/Closing Model Projects in Designer
>
> If users "Close" Model Projects that contain Source Model bindings to CF's on a Teiid instance, do we "filter" these CF's from the Teiid View? Would this require the concepts of "workspace" or "model project" to be included somehow in our CF properties?
>
> If a user re-opens a Model Project and there is NO info from their workspace (except maybe "model name") to re-bind their source models to CF's, how do we assist the user?
>
Use case:
>
> 1) Create relational model "MyRelationalModel" from scratch (maybe import from text file) in "MyOwnModelProject"
> 2) No JDBC info embedded in model (only if created via Import from JDBC)
> 3) Preview relational model - results in "Select/Create CF" for relational model and Hidden VDB creation for "MyOwnModelProject"
> 4) User Selects/Creates CF named "MyOwnTextConnectionFactory".
> 4) Model now previewable and bound to "MyOwnTextConnectionFactory" CF on Teiid instance
> 5) User closes Model Project and Exits Designer
> 6) User launches Designer with previous workspace
> 7) User Opens "MyOwnModelProject"
> 8) User selects table in "MyRelationalModel" and tries to "Preview Data"
>
> Questions:
>
>
> - When user "closes project" will the "hidden VDB" be undeployed from server? Seems it should be removed.
>
I think it should be removed. I think we should do our best to keep the Teiid server uncluttered.
> - If hidden VDB IS removed, then is the "MyOwnTextConnectionFactory" removed also? If so, then user has to re-create their "MyOwnTextConnectionFactory" from scratch.
>
Designer will have the capability to export the CF from Teiid, correct? We could document that for cases like this (text, xml, soap, salesforce, google spreadsheets) the user should export the CF into their model project. They could then import the CF back into Teiid when opening the model again.
> ISSUE 2: Switching Workspaces in Designer
>
> Eclipse provides a "Switch Workspace" feature which allows users to work on possible "different" sets of projects. If users are wiring multiple "workspaces" to the same Teiid instance, then there will be the potential for artifacts (VDB's, CF's) that may be shared between workspaces and some that won't.
>
> Are we gonna show all CF's to user? Maybe filter per workspace?
>
> If user doesn't ever re-open or delete first workspace, then CF's from that workspace would never get cleaned up. User's would have to use JON/JOPR to do that? From a usability standpoint, it seems awkward to tell Designer users to use another tool to "remove" objects created by and used solely by another tool.
Are you asking about concurrent connections from multiple workspaces here? I've read everything above with the understanding that what is deployed into Teiid from designer in terms of preview and VDB execution ( as opposed to actually Exporting a VDB into Teiid) was cleaned up when Designer was shut down. Perhaps a bad assumption on my part, but it is the way I think Designer should function. I think we should aggresively undeploy from Teiid.
Operating from that assumption, if I switch from workspace X to workspace Y, everything from X would get undeployed as Designer switched. If X wanted to share a CF or VDB, then the CF would have to be exported, shared with Y, and imported by Y. A VDB would have to be actually deployed to be shared.
This assumption however doesn't free us from my use of two workspaces at the same time, or a shared use of a Teiid server in any form. Even if we did filter per workspace, we could end up with clashes. What if you created a CF called MyCF, and I tried to do the same. Because we've filtered I can't see yours, but I fail to create mine.
>
> ISSUE 3: General CF management
>
> How do we handle CF's for VDB's that DO allow execution, but Designer DOESN'T allow preview? Note JDoyle's comment: The XML-Relational connectors have never worked with preview, you are required to do a join to get results. Is it possible to introduce the concept of a model or ConnectionFactory that does not support preview? It causes confusion for users when they try to preview and it fails.
>
> The question is still on the table whether or not Designer should just expose the basic CF management features of "Create/Edit/Delete/Import/Export". Many of the issues that have and probably will come up involve "how do we do this for the user, how do we hide this from the user and how do we fix this for the user". Allowing for general CF management could actually simplify things from a user standpoint. If we do "open the floodgates", so to speak, at least the user will "know" that no matter whether they view "Teiid" from JON or Designer, the CF's that exist are THEIR responsibility. They create, edit, delete, move etc... That actually seems the cleanest solution.
I'm in favor of opening the floodgates.
ISSUE 4: VDB Artifact Portability
>
> Currently, Designer doesn't provide an "Import VDB" feature. There are potential XML Schema issues when dealing with the old "Enterprise" datatypes that prevent this.
>
> However, with the re-factoring of the Connectors framework due to Teiid JCA changes, it makes it much more attractive and almost a requirement to allow "Importing" VDB's and exploding them into the workspace. Do we want to tackle this? Create a JIRA for 7.1? Is the "Enterprise" datatypes issue solvable?
>
>
> Barry
>
> _______________________________________________ teiid-designer-dev mailing list teiid-designer-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/teiid-designer-dev
14 years
Teiid Designer 7.0 Release Planning
by Barry Lafond
The following is a culmination of various discussions over the past month around what Teiid Designer 7.0 should look like or how it should be behave based on fundamental changes to the Teiid server. These changes mainly affect Admin API and migration to using the JCA compliant container technology compatible with JBoss AS.
These changes have required us to re-architect Designer's connector management framework as well as a rewrite the VDB artifact creation, editing and management framework.
Note that the new target "platform" for Designer includes integration with JBoss Tools plus the availability of JOPR for Teiid Server management.
Please view these plans and all feeback and suggestions are welcome.
Barry LaFond
Teiid Designer Project Lead
===============================================
Basic Functionality
A) Designer is a Teiid design tool required for creating consumable/deployable VDB's containing indexed/queryable models.
B) Designer is NOT a Connector/Connection Factory (CF) management tool. (JOPR/JON is)
- Note that in Designer 6.0 thru 6.3 Connectors were refered to as "Connector Types" and Connection Factories were called "Connector Bindings". Designer 7.0 will be adapting the new terminology.
C) Designer will provide a Teiid View (name TBD) which exposes viewable/filterable content of one or more "Teiid Servers". (Similar functionality to Release 6.3's Connectors View)
- Designer allows connecting to and viewing contents of multiple Teiid Servers
- Shows Connectors as children of Server
- Shows CF's as children of Connectors
- Shows "Source Models" (bindings) as children to CF's
- Bind/Unbind features remain similar to 6.3 release
- NO CF info is persisted within Designer. All of it is persisted and accessed through Teiid.
- View will provide "filters" to hide Connectors if so desired. (current 6.3 feature)
- View will provide "pivot" feature to show "SourceBinding-centric" view (Stretch goal)
- View will support the notion of a "Default Teiid Server". Preview Data and VDB development will be tied to the current "Default Teiid Server".
- Users can change their "Default Teiid Server" at any time
• Not sure how Designer reacts to switching "Default Teiid Server"
• Could allow migration of CF's from old Default to New Default
• ... or assume user will take care of it?
D) Designer will assist when needed to help user create/select CF's to either Preview Data or construct deployable VDB's.
- Any Relational model created via JDBC import that contains JDBC Import info will be a candidate for auto-discovery of existing CF.
- All other Relational models will required user-intervention to select an existing CF or create a compatible CF from existing Connector (i.e. type)
- CF's are named with respect to the corresponding bound "Model name"
- CF's created from Designer are managed from Designer and are "Model-centric". If a model is deleted from Designer, the corresponding CF is deleted from server.
- Users can "Edit" a CF from Designer to change basic properties, reset password, etc...
- Users cannot Create CF from Designer unless directed to via "Preview Data" or VDB editing.
E) Designer will NOT perform auto-CF creation or matching during JDBC import (ala MM 5.5.3 & pre Teiid Designer 7.0)
- The reasoning behind this change is a result of the role that Designer is now targeted to fill within the JBoss Tools enviroment, which is to create and test artifacts for deployment to Teiid Servers.
- Instead of assuming that a users will always want to "Preview Data", for instance, we are now assuming that the may want to, so we are delaying the CF management part until actually needed.
F) Designer will implement a "Hidden VDB" framework to allow Preview Data functionality.
- Preview Data functionality was provided "Pre-7.0 Designer" via separate embedded Teiid code. Maintenance of this code has been expensive, so to speak, so removing that requirement has been a goal of Teiid & Teiid Designer.
- The simplicity and flexibility of Teiid's ability to deploy VDB's has provided a mechanism to replace that embedded Teiid source code with a deployed "Hidden VDB" which is virtually 100% compatible with the VDB execution feature which is already a Designer requirement.
- Bindings of "Source" models to CF's will be displayed in Designer's Connectors View (Name of view TBD... maybe Teiid? Execution?)
- This hidden vdb will not be exposed to the user.
- Hidden VDB will be created/deployed when "Preview Data" is initially requested.
- Hidden VDB will be "Teiid-Server Centric" (i.e. one per server)
===============================================
Related issues
ISSUE 1: Opening/Closing Model Projects in Designer
If users "Close" Model Projects that contain Source Model bindings to CF's on a Teiid instance, do we "filter" these CF's from the Teiid View? Would this require the concepts of "workspace" or "model project" to be included somehow in our CF properties?
If a user re-opens a Model Project and there is NO info from their workspace (except maybe "model name") to re-bind their source models to CF's, how do we assist the user?
Use case:
1) Create relational model "MyRelationalModel" from scratch (maybe import from text file) in "MyOwnModelProject"
2) No JDBC info embedded in model (only if created via Import from JDBC)
3) Preview relational model - results in "Select/Create CF" for relational model and Hidden VDB creation for "MyOwnModelProject"
4) User Selects/Creates CF named "MyOwnTextConnectionFactory".
4) Model now previewable and bound to "MyOwnTextConnectionFactory" CF on Teiid instance
5) User closes Model Project and Exits Designer
6) User launches Designer with previous workspace
7) User Opens "MyOwnModelProject"
8) User selects table in "MyRelationalModel" and tries to "Preview Data"
Questions:
- When user "closes project" will the "hidden VDB" be undeployed from server? Seems it should be removed.
- If hidden VDB IS removed, then is the "MyOwnTextConnectionFactory" removed also? If so, then user has to re-create their "MyOwnTextConnectionFactory" from scratch.
ISSUE 2: Switching Workspaces in Designer
Eclipse provides a "Switch Workspace" feature which allows users to work on possible "different" sets of projects. If users are wiring multiple "workspaces" to the same Teiid instance, then there will be the potential for artifacts (VDB's, CF's) that may be shared between workspaces and some that won't.
Are we gonna show all CF's to user? Maybe filter per workspace?
If user doesn't ever re-open or delete first workspace, then CF's from that workspace would never get cleaned up. User's would have to use JON/JOPR to do that? From a usability standpoint, it seems awkward to tell Designer users to use another tool to "remove" objects created by and used solely by another tool.
ISSUE 3: General CF management
How do we handle CF's for VDB's that DO allow execution, but Designer DOESN'T allow preview? Note JDoyle's comment: The XML-Relational connectors have never worked with preview, you are required to do a join to get results. I s it possible to introduce the concept of a model or ConnectionFactory that does not support preview? It causes confusion for users when they try to preview and it fails.
The question is still on the table whether or not Designer should just expose the basic CF management features of "Create/Edit/Delete/Import/Export". Many of the issues that have and probably will come up involve "how do we do this for the user, how do we hide this from the user and how do we fix this for the user". Allowing for general CF management could actually simplify things from a user standpoint. If we do "open the floodgates", so to speak, at least the user will "know" that no matter whether they view "Teiid" from JON or Designer, the CF's that exist are THEIR responsibility. They create, edit, delete, move etc... That actually seems the cleanest solution.
ISSUE 4: VDB Artifact Portability
Currently, Designer doesn't provide an "Import VDB" feature. There are potential XML Schema issues when dealing with the old "Enterprise" datatypes that prevent this.
However, with the re-factoring of the Connectors framework due to Teiid JCA changes, it makes it much more attractive and almost a requirement to allow "Importing" VDB's and exploding them into the workspace. Do we want to tackle this? Create a JIRA for 7.1? Is the "Enterprise" datatypes issue solvable?
Barry
14 years
Milestone today
by Steven Hawkins
Hello all,
It looks like we should be good to use last night's build (a concurrency issue was addressed with the buffermanager) as the milestone. We'll work on getting it posted today.
Thanks to everyone,
Steve
14 years
Fwd: Connection Factories
by Barry Lafond
One thing I'd like to keep in the back of our minds... is the perception that dealing with a bunch of xxxxx.xml files is the "Norm" when it comes to JBoss Tools
AND.....
There has been lots of "bad" buzz around it. One of the knocks about JBDS/JBT is still the hand-editing of xml files... and managing xml files.
It's not like we're gonna build XML Text editors for Eclipse for Designer. We have the hooks to already Edit (via property panels), what amounts to "-ds.xml" files through the API.
Teiid Tooling is what we are. It's what we do. Teiid Designer needs to provide an attractive user experience. Maybe if that means duplicating some JON-type features to make that happen, so be it.
Barry
----- Original Message -----
From: "Ramesh Reddy" <rareddy(a)redhat.com>
To: "John Verhaeg" <jverhaeg(a)redhat.com>
Cc: "teiid-dev" <teiid-dev(a)lists.jboss.org>
Sent: Friday, April 9, 2010 12:26:30 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] Fwd: [teiid-designer-dev] Connection Factories
On Fri, 2010-04-09 at 11:25 -0500, John Verhaeg wrote:
> Do we have to include *-ds.xml files to accomplish this? Thus far, we
> haven't been planning on doing this...
No, I am not saying to include this. If we can export CF, that is one
way to share. Or user1 can go the deploy directory of the container to
grab the -ds.xml files.
Ramesh..
_______________________________________________
teiid-dev mailing list
teiid-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/teiid-dev
14 years
Re: [teiid-designer-dev] Connection Factories
by Barry Lafond
Removing the ability of a user to perform general CF management from Designer is the main sticking point here.
I agree with John that to be consistent with the other pieces of the SOA experience (aka JON/JOPR), then keeping this feature in Designer would be duplication of effort.
I don't know if I'd remove it just based on the "... well find me a valid use case" card. Trust me, if we asked that question for all of Designer features, we'd be yank ing a bunch of other things too.
Though I posted the "plan" and John very elegantly explained it in detail (THX JPAV), right now the proposal is on the table and we haven't yanked the CF creation/editing framework just yet.
IF there is ANY use-case involving "Developers" to work with MULTIPLE Vdb's and MULTIPLE Teiid instances from DESIGNER, then I'd lean more toward keep CF management feature in. Ramesh points out some possible ones that do make sense. Designer is going to have the API to implement this feature so we'd have to deliberately not expose it.
Any other perspectives?
Barry
14 years
Connection Factories
by Barry Lafond
Per recent discussions and decisions about our Connector framework I want to clarify our current direction for Designer. So please decipher the following and speak up now.
=========================
A) We are proposing removing the notion of general "editing/managing" Connection Factories (formerly known in Designer as Connectors).
B) This would mean that users would NOT be able to select Connectors (Types) and create new instances of them (Connection Factories) NOR edit existing instances on a given Server. That responsibility will be on the shoulders of JON.
C) For the purposes of "Preview Data" and even VDB Execution, Designer WILL assist the process of matching/binding sources to existing Connection Factories using any JDBC Source/URL information available within source models.
- During JDBC Import, the URL information will be used to try and match /select an existing Server Connector &/or Connection Factory
- If an existing Connection Factory does not exist AND the URL information sufficiently matches the properties for a Connector (TYPE), then a Connector Factory will be created and the source binding created.
- If no Connection Factory can be matched/created user can Select One (which may or may not work) or the Source cannot be previewed or executed in a VDB.
=========================
If you disagree with this functionality or have any questions, please respond.... otherwise we'll be gutting a bunch of UI code in the next day or so :)
Barry LaFond
Teiid Designer Project
14 years
Re: [teiid-designer-dev] [teiid-dev] query plan changes
by Steven Hawkins
Yes the defaults are both off. The SET command is scoped to the session and currently has the same affect as changing the connection or datasource properties. Using NOEXEC is the same as Sybase and SQLServer, which also use the EXEC syntax for calling procedures. We can of course make it whatever we want.
----- Original Message -----
From: "Ken Johnson" <kejohnso(a)redhat.com>
To: "Steven Hawkins" <shawkins(a)redhat.com>
Cc: "teiid-users" <teiid-users(a)lists.jboss.org>, "teiid-dev" <teiid-dev(a)lists.jboss.org>, "teiid-designer-dev" <teiid-designer-dev(a)lists.jboss.org>
Sent: Tuesday, April 6, 2010 2:10:59 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] query plan changes
Steven Hawkins wrote:
> Hello all,
>
> Currently we have OPTION SHOWPLAN PLANONLY DEBUG to control planning and execution. The drawbacks are that this introduces three reserved words (SHOWPLAN, PLANONLY, DEBUG) and it's only applicable to the top level user query.
>
> The proposed replacement makes use of https://jira.jboss.org/jira/browse/TEIID-906, which added basic support for a set statement:
>
> SET SHOWPLAN [ON|DEBUG|OFF]
> SET NOEXEC [ON|OFF] - general replacement for PLANONLY, will not execute the command if ON
>
This seems a reasonable way to reduce the keywords and allow repetitive
executions without having to modify the query syntax each time. How are
these commands scoped, per session? It would be cool if these could be
used to enable plan output for a query coming from an application where
the user cannot easily change the query itself. Presumably defaults
would be SHOWPLAN=OFF and NOEXEC=OFF.
Given that EXEC is a reserved word for executing a procedure, do you
think NOEXEC might cause confusion (users thinking it's the negative
form of EXEC)?
> I would also like to remove most of the logic surrounding the org.teiid.client.plan.PlanNode class. The visitors, display helpers, etc. Since we don't publish a meaningful schema for the xml form(s) or document the property keys we might as well assume that we're the primary consumers and forgo the flexibility.
>
Don't know about this one. Hard to foresee how users might consume the
query plan XML in the future.
> Any thoughts?
>
> Steve
>
> _______________________________________________
> teiid-dev mailing list
> teiid-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
>
--
Ken Johnson
Sr. Product Manager
JBoss Middleware Business Unit
Red Hat, Inc
978.392.3917
ken.johnson(a)redhat.com
14 years
query plan changes
by Steven Hawkins
Hello all,
Currently we have OPTION SHOWPLAN PLANONLY DEBUG to control planning and execution. The drawbacks are that this introduces three reserved words (SHOWPLAN, PLANONLY, DEBUG) and it's only applicable to the top level user query.
The proposed replacement makes use of https://jira.jboss.org/jira/browse/TEIID-906, which added basic support for a set statement:
SET SHOWPLAN [ON|DEBUG|OFF]
SET NOEXEC [ON|OFF] - general replacement for PLANONLY, will not execute the command if ON
I would also like to remove most of the logic surrounding the org.teiid.client.plan.PlanNode class. The visitors, display helpers, etc. Since we don't publish a meaningful schema for the xml form(s) or document the property keys we might as well assume that we're the primary consumers and forgo the flexibility.
Any thoughts?
Steve
14 years