[teiid-designer-dev] [teiid-dev] VDB Versioning Feature

Steven Hawkins shawkins at redhat.com
Wed Apr 21 17:03:55 EDT 2010 

Before responding to you're conclusions below, I'd like to bring up that we currently report the vdb name as the catalog name through our system tables / jdbc metadata.  So to remove vdb version we would either need to:
- not report the vdb name as the catalog to prevent apparent behavioral differences 
- or change the vdb name in the vdb.xml to mean the catalog name

I assume we would take the latter approach.  In that world creating a copy of a vdb does not require modifying the vdb.xml, just creating a new file.  However we still need a unique way to refer to each vdb, so I would propose using the filename (designer already starts off with the assumption filename = vdbname).  While it's possible through direct file deployments to do all kinds of crazy things (vdbs inside of zip files, sub folders, etc.), we can assume we're really only interested in this discussion with our "managed" view of the deploy directory.  In which case we effectively do control the .vdb and -vdb.xml artifacts there.

To your points:

a) We could forgo even adding a copy vdb method if we put versioning as a primary concern in the deployVDB admin method.  There are a couple of possible variants:

- overwrite the previous
- give a different filename to the vdb to deploy (pseudo active default behavior)
- give a different filename to the old vdb (which would be nothing more than a combined copy then deploy operation)

The current admin api method for vdb deployment can handle the first two cases - void deployVDB(InputStream contents, String fileName).  It would probably make more sense in the last case for the admin to specify the name rather than to use auto-increment logic since a proper name allows you to know in advance how to change your urls.  This means that we only have to add 1 additional method - boolean deployVDB(InputStream contents, String fileName, String oldVdbName) returning true if the old file existed.  The associated tooling would then need to have deployment screens that specify if this deploy is an overwrite or what the backup name should be.
      
b) Other than keeping the vdb name / file name the same between deployments, there wouldn't be deployment related versioning concerns at design-time.

c) Seems like this follows from a.

----- Original Message -----
From: "Ken Johnson" <kejohnso at redhat.com>
To: "Steven Hawkins" <shawkins at redhat.com>
Cc: "Larry O'Leary" <loleary at redhat.com>, "teiid-designer-dev" <teiid-designer-dev at lists.jboss.org>, "teiid-dev" <teiid-dev at lists.jboss.org>
Sent: Wednesday, April 21, 2010 12:49:50 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] [teiid-designer-dev]   VDB Versioning Feature

I think this one has been flogged pretty well but I'd like to take 
another crack at defining an alternative to the current version 
number-based approach.

The principle value of the exising VDB versioning is as Ramesh 
summarized: Client Transparency. We want a way for administrators to 
introduce new versions of VDBs and seamlessly migrate clients that are 
prepared to migrate as well as provide a simple means to allow laggard 
clients to continue using an older/deprecated version of the VDB.

It seems this can be achieved largely through naming convention and good 
process, with support from APIs/tools. From a user experience point of 
view, the old solution had numerous benefits:

1. auto-version-increment prevented accidental overwrites on deployment 
and eliminated the need for convention or best-practice
2. the name+ version number approach made it clear to administrators 
that a set of VDBs are really the same thing but at different revision 
levels.
3. the active-default allowed administrators to control the vdb used by 
clients without disrupting the client code/configuration and didn't 
break active client connections.

The replacement solution, though different in implementation, should 
strive to have a similar level of user-friendliness:

Deployment via AdminAPI/shell and admin-console provide an opportunity 
to constructively guide the user. For example warning them when 
deploying a VDB with the same name as a currently deployed VDB. Or 
giving them an option to rename the existing VDB to a "backup" name 
prior to deploying the new version. We should help the user to not shoot 
themselves in the foot. Deployment via file drop/copy doesn't afford 
much opportunity to help users.

Best practices/scenarios should be documented so users understand how to 
manage VDB "versions", not just in production but across the various 
lifecycle stages (dev->test->prod). One usage scenario is for ModeShape 
to be used as a repository of artifacts like VDB where administrators 
can locate the "approved-for-deployment" VDB and then deploy it via URI 
using AdminAPI/shell or admin-console. The best practices should provide 
guidance here as well.

So, IMO option 1 can work if:
a) API and tooling operations are added to assist/protect the user
b) the Design-time developer is not responsible for anything related to 
version management. This is an administrator activity.
c) most importantly: This continues to be offered as a feature. By that 
I mean, this user need is acknowledged and Teiid provides 
facilities/docs that address "VDB version management" as a capability 
(even if multi-step) and not just "something a user can do if so inclined"

Thoughts?
-Ken


Steven Hawkins wrote:
> Is there any consensus here?  Mike, Ken?
>
> As I see it:
>
> Option 1: get rid of the version concept altogether and possibly add a rename or copy vdb operation.
>
> Option 2: retain the version concept along with the active default designation.  However setting the version will be a design time responsibility and file drop deployments will require different vdb file names for different versions.
>
> Option 3: ?
>
> And regardless of the scenario, deployers will need to ensure that only truly active vdbs are in an active state or suffer the memory/load costs.
>
> ----- Original Message -----
> From: "Larry O'Leary" <loleary at redhat.com>
> To: "Steven Hawkins" <shawkins at redhat.com>
> Cc: "teiid-designer-dev" <teiid-designer-dev at lists.jboss.org>, "teiid-dev" <teiid-dev at lists.jboss.org>, "John Verhaeg" <jverhaeg at redhat.com>
> Sent: Thursday, February 18, 2010 4:13:03 PM GMT -06:00 US/Canada Central
> Subject: Re: [teiid-designer-dev] [teiid-dev]  VDB Versioning Feature
>
> On Thu, 2010-02-18 at 15:43 -0500, Steven Hawkins wrote:
>   
>> At first glance in the scenario you list below, there's no need to
>> even bother with the promotion concept.  You would just install the
>> new version (with a new name) and let apps point over to it as they
>> are ready.
>>     
>
> Well, except for the fact that the clients do not want to change their
> connection properties.  In other words, whatever the business as a whole
> deems as the latest and greatest, they are fine with.  It is just that
> there is normally some stragglers that do things their way and disregard
> the business decision.  In these cases, these stragglers would be
> responsible for updating their client connection properties to point to
> the renamed (previous version) VDB.  Must like in legacy, they would
> update their connection properties to point to the old version. 
>
>   
>> To achieve the flow your suggesting without versioning would actually
>> require an extra step.  Prior to promoting the next version, you would
>> need to deploy a duplicate of the current version with a new name.
>> Then the small set of apps not ready for the update would need to have
>> their connections pointed to the copy.  Then you could safely promote
>> the next version without interrupting the lagging apps.
>>     
>
> Right.  In this scenario the new VDB would get the name of the old VDB
> so that client apps that were following the business decision would not
> need to make a change to get the newer VDB.  And the lagging users would
> update their clients to point to the renamed VDB.  
>
> I am not familiar with how we are packaging and naming VDBs to achieve
> this but in my opinion, making this a manual process seems logical and
> there would be no need for the system to automatically keep track of
> things.
>
> Furthermore, the functionality of "active-default" can be achieved the
> same way.  Just make the active-default or latest version a base name.
> MyCoolSource.vdb.  When a new version is available for integration
> testing but you are not ready to promote it to the "active-default" or
> "latest-version", deploy it as MyCoolSource_NextGen.vdb.  In this case,
> users who want to use the NextGen version would simply change their
> client app to point to MyCoolSource_NextGen.vdb and everyone else will
> continue to use the "active-default" of MyCoolSource.vdb.  
>
> When you are ready to move the "active-default" to the contents of
> MyCoolSource_NextGen.vdb then you simply copy MyCoolSource.vdb to
> MyCoolSource_OldGen.vdb, copy MyCoolSource_NextGen.vdb to
> MyCoolSource.vdb and voila.  The only downside here is that
> MyCoolSource.vdb and MyCoolSource_NextGen.vdb are duplicates but I
> really don't see this as a problem.  
>
> The naming convention would be up to business rules and we might want to
> make some kind of best practice suggestions to make this manageable but
> concept could be just as easily accomplished with version numbers in the
> name:
>
> MyCoolSource_v1.vdb
> MyCoolSource_v2.vdb
> MyCoolSource_v3.vdb
>
> And to achieve "active-default" or "latest-version" you would use
> MyCoolSource.vdb -> MyCoolSource_v3.vdb as a symbolic link or copy of
> the version you wanted to use.
>
> Of course, this logic all falls apart if the VDB actually contains a
> descriptor that defines its name to the system in which case the process
> is the same but a bit more complicated because you wouldn't be able to
> simply copy/symlink.
>
>
>   
>> ----- Original Message -----
>> From: "Larry O'Leary" <loleary at redhat.com>
>> To: "John Verhaeg" <jverhaeg at redhat.com>
>> Cc: "teiid-designer-dev" <teiid-designer-dev at lists.jboss.org>, "teiid-dev" <teiid-dev at lists.jboss.org>
>> Sent: Wednesday, February 17, 2010 11:38:00 AM GMT -06:00 US/Canada Central
>> Subject: Re: [teiid-designer-dev] [teiid-dev]  VDB Versioning Feature
>>
>> On Wed, 2010-02-17 at 11:27 -0600, John Verhaeg wrote:
>>     
>>> On Feb 17, 2010, at 10:44 AM, Larry O'Leary wrote:
>>>
>>>       
>>>> Well there are two possibilities here.  One is that there is a
>>>> roll-back.  The other is that a new version of a VDB has been chosen as
>>>> the default but some of the organization is not ready to use the newer
>>>> version.  In which case they would modify their clients to use the
>>>> previous version until they are ready to use the new one.  
>>>>
>>>> So, in the "roll-back" scenario, the change is transparent to the
>>>> clients while in the "not-ready-to-upgrade" scenario the administrative
>>>> change is placed on the client.  These are the two cases we have with
>>>> legacy customers.
>>>>         
>>> Maybe I'm misunderstanding, but I don't believe your "new version"
>>> scenario makes practical sense.  If an organization is implementing a
>>> new version and not ready to force that change on certain
>>> applications, departments, etc., wouldn't they want just that - no
>>> change to those areas?  They're likely not going to impose the need to
>>> change a slew of existing clients to use the old VDB just to enable
>>> new clients to use the previous VDB name.  Rather, they'd want to
>>> deploy the new VDB with a new name that only the new clients use.
>>> Then migration to the new VDB name for other clients could be easily
>>> scheduled and managed.  Seems like the alternative would be a
>>> change-control nightmare.
>>>
>>>       
>> Here is how it works.  All clients are using an existing VDB and
>> development creates a new version of that VDB.  They put it through all
>> the testing and identify any possible conflicts and changes that a
>> client may need to make.  The VDB is then slated to be migrated to
>> production during their quarterly maintenance window.  However, a small
>> group within the organization have not been able to complete all of
>> their testing necessary to use the new version of the VDB.  In this
>> case, a business decision is made to proceed with the migration into
>> production and the "small group" is notified that they have until MM/dd
>> to complete their testing or they will need to point their clients at an
>> old version of the VDB.  The old version will be left in production but
>> as a deprecated source.
>> --
>>
>> Larry O'Leary
>> Middleware Support Engineering Group
>> Global Support Services
>> Red Hat, Inc.
>> loleary at redhat.com
>> 1.866.LINUX70 (+1.314.336.2990) xt 81-62909
>> Office: +1.314.336.2909
>> Mobile: +1.618.420.7623
>> --
>> Looking to carve out IT costs?
>> http://www.redhat.com/carveoutcosts/
>>
>> _______________________________________________
>> teiid-designer-dev mailing list
>> teiid-designer-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/teiid-designer-dev
>>     
>
>

More information about the teiid-designer-dev mailing list