[teiid-dev] 6.3 planning

[Steven Hawkins]

There hasn't been much on the dev or user list about work for 6.3, but it's time again to start being more formal about our expectations for the next release cycle.  There are three main engineering goals for the 6.3 release cycle (~12 weeks):

1. To refactor caching/simple materialization.

This entails removing source result set caching (TEIID-168), retargeting user result set caching to use BufferManagement (this removes the memory-only restriction, and is more efficient since we already have the result captured in a tuplesource), to automatically scope cache results based upon the determinism level of the command, to create implicitly and explicit (through metadata or language directives) relation caches (which would be basically equivalent to temp tables) and to update planning to take advantage of the cached tables.  Much of this is already discussed in TEIID-168, but needs expanded upon.  The usage of Infinispan should be preferred as a replacement for our usage of JBoss cache given its touted performance improvements and better messaging of Teiid as "cloud ready".  Infinispan could also be targeted as a backend for the BufferManager/temp tables (there would be an opportunity to address the temp table defects TEIID-189 TEIID-89 TEIID-162).  And last, but not least, code tables should be reworked (TEIID-829)

2. Provide a deeply integrated container deployment mode.

This entails changes to our transaction handling and threading to allow for usage of a container supplied transaction manager (Needs a JIRA).  Beyond that, Ramesh is investigating integrating Teiid as a JCA component - which can handle both embedded and server mode transaction/threading/security and JON discovery.  The JBoss JCA project provides a simplified MC based deployment environment, that would serve as the basis for a replacement of standalone server deployment.  Taken to a logical extreme we would also want to abstract the transport handling out of the XML/Salesforce connectors (Needs a JIRA) and rely on more container provided mechanisms.

3. Add a kit download for a "usable" JOPR console - Needs a JIRA, and continue ensuring adminapi/admin shell coverage of all exposed functionality.

As with 6.2, this work should have minimal impact on Designer's usage of Teiid embedded.  Any new work in Designer would be a stretch goal only.

The engineering stretch goals for this release are:

1. Have definitive plans around Teiid owned metadata in a persistent form, which has already been on introduced as a topic on the dev-list.

2. Re-implement file and database based artifact access through JCR (DNA) - with an eye toward using JCR as a means to provide "cluster" features, but that would still be api compatible with the embedded implementation (JCR backed by the filesystem).

3. Allow for pass-through ddl in both the single source and multi-source scenarios - TEIID-669

4. Name-spaced/connector scoped functions to allow for broader function push-down.  This could also be tied to supporting import and exposing JDBC 4.0 style function metadata (Needs a JIRA) See also (TEIID-502 TEIID-231)

5. Runtime costing update (TEIID-245)

6. Sub-query optimizations (TEIID-211)

... and many more that should be upped in priority ...

We should keep in minde that a 6.4 bucket may be needed depending upon the intermediate head way made on 7.x

The remaining 6.2 doc issues around our legacy SOAP services and some proposed REST services in terms of how they can be implemented with JBoss projects with a loose coupling to Teiid have been moved to 6.3.

On the non-engineering side of things we have the requirements of:

1. Updating the server extension guide to cover things from an embedded/simplified perspective given the changes to command logging, membership domains, etc. (slipped from 6.2)

2. Add the jdbc developers guide - TEIID-315 (slipped from 6.2)

The on-going ease of use issues are:

1. Adding additional screen casts covering a broader set of common uses.

2. Add more examples to the kit, especially highlighting use of non-relational sources.

3. Get Teiid as a recognized source in other products/projects - squirrel, dtp, hibernate, etc.

Please provide feedback on missing items or those that need more detail.

Note there are missing JIRAs for a lot of these items and the priority may not yet be assigned correctly for items identified as requirements.  Devs and leads should feel free to help fill in gaps.  Let me know if there is anything that has been left out or what we should dive into a little more detail on.

The Teiid Team