[teiid-dev] Translators in the "vdb.xml" file.

Tue May 25 16:22:36 EDT 2010

----- Original Message -----
From: "Barry Lafond" <blafond at redhat.com>
To: "Steven Hawkins" <shawkins at redhat.com>
Cc: teiid-dev at lists.jboss.org, "Van Halbert" <vhalbert at redhat.com>
Sent: Tuesday, May 25, 2010 2:20:41 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] Translators in the "vdb.xml" file.

geesh......... here goes....some random thoughts and questions.... 

1) If you are proposing embedding custom Translator XML into vdb.xml , wouldn't that info be worthless unless the JAR file came with it in the VDB? 

> The xml is not custom - the property sets may be (<property name="foo" value="bar"/> vs. <property name="foo1" value="bar"/>).  I'm not sure what your perspective is with the worthless comment.  Is a -ds.xml worthless without the corresponding .rar?  

2) Designer is not intended to be a Translator creator/editor framework. 

> Nor are you being asked to be.  You are already creating translator specific property sets, all we're saying is that there's a better place for that information.

3) Isn't the CDK providing functionality for creating/editing Translators for custom connectors.....and isn't it the primary tool for maintaining all global "connector" entities specific to Teiid? 

> I'm not even sure what you are asking here, see the answer to 1) and 2) you may have the wrong idea about what's being proposed.

4) From a Designer perspective, the user really only sees/knows about Data Source connection info like: driverClass, vendor, version, databaseName, URL, username and password. That's all the info they have, or required to have to import JDBC. 

> You may not require a translator as this point, but you are implicitly creating metadata that will function with the corresponding translator.

5) Not sure how Designer is moving down the path where "physical model metadata is tied directly to the translator used". We import JDBC metadata and "tweak" some datatypes to match our relational metamodel datatypes, but basically the Relational Source model is a structural representation (with properties) right? If the "translator" properties include items such as xa-capable, immutable, max-results-rows, and execution-factory-class, I don't really see much in the way physical model metadata. I did look at some of your ExecutionFactory classes and I see lots of Database-specific "translation" logic/constants, but none of this gets exposed to Designer, right? 

> Most definitely you have moved down that path.  Any of the non-JDBC importers rely on extension metadata that is useful only to its corresponding translator.  In some cases base metadata is overloaded, so for example an XML/Relational table may have the name in source of '/' - which conveys the XPath and is completely different than how name in source is used for JDBC.  As for the base properties you mention, xa-capable will probably go away.  There is only 1 case where it's even useful and that's to compensate for a bug in the mysql driver.  Likewise for immutable, now that we don't care from a translator perspective whether we are transaction aware or not, immutable as a translator property isn't useful.  max-results-rows could be done differently - it is typically only a development time aid and for most connectors doesn't actually prevent a runaway query.  It just limits how much Teiid processes.

As for what gets "exposed" to designer.  Directly nothing, since you don't have any dependencies on that logic.  However indirectly as stated in the response to 4) the metadata is tailored for the translator.  

The remaining translator properties fall into four camps:
properties that should not change from environment to environment - text encodings, jdbc trim strings, jdbc databasetimezone, etc.  These properties could have just as easily been seen as additional metadata on the model
properties that are possibly misplaced - xml logRequestResponseDocs (could be based upon a specific log context)
properties that are from legacy features - jdbc usecommentsinsourcequery 
properties that are related to performance - jdbc fetchSize, jdbc useBindVariables

Only the last set have any expectation of being needed/changing between environments in such a way as vdb redelopment seems like overkill.  In both of these JDBC cases we may be doing something wrong here.  Most applications, see hibernate for example, will just always generate prepared statements.  At the very least we should change the default to true and possiblely even drop the ability to set the value.  Also, fetchSize is being applied to all statements regardless of the size of the resultset, which makes it extremely dangerous to use when there is variability across source queries.

6) (see 5 above) Have no idea what "change your translator == change your physical model" means other than some cheer-leader reference :) 

> Without trying to understand what's being presented to you, yes you could come to that conclusion.

7) .. your statement "... native types, column names, etc., are not the same across all source types." Can you give me an example? ..............not sure what you are expecting Designer to do with these changes? I can see a "dynamic vdb" handling it, but our Relational Source models are instances or views of a specific DB schema (or other data source type). I don't think we have the notion of a "flexible" column name? 

> I'm not suggesting that you need a notion of a flexible column name / native type (although with non-homogeneous databases in a multisource mode we do have some problems).  A good example would be that an Oracle native type of date is semantically different than a SQL Server date, which can be seen in the initial type mapping (Oracle will be to our timestamp).  Having a model imported from Oracle implies that it should only be used with the Oracle translator.  You may have thought of all this only from a modeling perspective - I've imported stuff and now I have a bunch of metadata.  Indeed in that view of the world the metadata is completely detached from it's execution.  However free standing metadata is not really the intent of Teiid Designer. 

Barry 

----- Original Message ----- 
From: "Steven Hawkins" <shawkins at redhat.com> 
To: "Van Halbert" <vhalbert at redhat.com> 
Cc: teiid-dev at lists.jboss.org 
Sent: Tuesday, May 25, 2010 1:31:39 PM GMT -06:00 US/Canada Central 
Subject: Re: [teiid-dev] Translators in the "vdb.xml" file. 

This is a change toward simple and focused. With the environmental information held by a resource adapter connection factory the remaining properties of a translator are not expected to change. It simplifies deployment greatly to have the vdb be self contained than maintaining translators as another global entity. 

In any event the deployment of a custom translator is also greatly simplified from before, and should now become as easy as placing the JAR with your ExecutionFactory in the deploy directory. 

If a custom translator is changed (same name/class, but with different property definitions), then the expectation is that the developer either has considered the impact of the migration, such that the old translators property sets are compatible with the new. If a new name is given and the old custom ExecutionFactory is removed, then a forced migration would occur that requires the change of affected vdbs. 

I don't necessarily see that as a limitation. We are steadily moving down the path where from dynamic vdb or even designer perspective the physical model metadata is tied directly to the translator used. This is especially true for LDAP and Salesforce, but is even true for JDBC as native types, column names, etc., are not the same across all source types. Only the loopback translator is a jack of all trades, but is of limited usefulness for anything other than basic queries. 

So to state this a different way, change your translator == change you physical model. 

----- Original Message ----- 
From: "Van Halbert" <vhalbert at redhat.com> 
To: "John Verhaeg" <jverhaeg at redhat.com> 
Cc: teiid-dev at lists.jboss.org 
Sent: Tuesday, May 25, 2010 1:11:14 PM GMT -06:00 US/Canada Central 
Subject: Re: [teiid-dev] Translators in the "vdb.xml" file. 

I vote the designer doesn't push the creation of a translator by adding additional translator properties to the vdb.xml file. If the user created a custom connector, they'll already be dealing with the task of deploying it to server(s). Also, if this information is stored in the vdb (call it A) and then if the original custom translator is changed (and replaced) and then vdb "A" is redeployed, don't we now have a usability issue that has to be resolved on whether to replace the existing translator properties or not. Carrying stale information in the vdb has caused us problems in the past, so I would hope we wouldn't head back down that road on the first release. 

I think this first release should be simple and focused. If the usability in the next release drives it in this direction, then so be it. 

On May 25, 2010, at 12:39 PM, John Verhaeg wrote: 

> On May 25, 2010, at 11:21 AM, Ramesh Reddy wrote: 
> 
>> What I forgot mention in the earlier is, if user chooses one of the 
>> default "translators", then there is not really any need to define the 
>> XML fragment for the Translator at all inside the vdb.xml. Which I 
>> suspect in majority of the cases. Only when user wants to alter the 
>> default behavior, Teiid would need the additional translator fragment. 
> 
> 
> So presumably there will be some property on the translators that indicates whether its a default translator? 
> 
> Thanks, 
> 
> JPAV 
> 
> 
> 
> 
> _______________________________________________ 
> teiid-dev mailing list 
> teiid-dev at lists.jboss.org 
> https://lists.jboss.org/mailman/listinfo/teiid-dev 

Van Halbert 
Principal Software Eng. 
Red Hat, Inc. 
------ 
vhalbert at redhat.com 

_______________________________________________ 
teiid-dev mailing list 
teiid-dev at lists.jboss.org 
https://lists.jboss.org/mailman/listinfo/teiid-dev 
_______________________________________________ 
teiid-dev mailing list 
teiid-dev at lists.jboss.org 
https://lists.jboss.org/mailman/listinfo/teiid-dev 