Re: [teiid-dev] XML as a source
by Steven Hawkins
John,
Option 2 is essentially what the xml connector does - joining information from a parent to a child element would cause two passes over the response document, correct? Option 3 is more work and would be less expressive that option 2 given that it's using sql constructs and not xpath.
The next question is what would document retrieval look like?
----- Original Message -----
From: "John Doyle" <jdoyle(a)redhat.com>
To: "Steven Hawkins" <shawkins(a)redhat.com>
Cc: "teiid-dev" <teiid-dev(a)lists.jboss.org>, "Ramesh Reddy" <rareddy(a)redhat.com>
Sent: Thursday, September 17, 2009 3:52:14 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] XML as a source
Option 2 seems like an attractive solution, I'm of the opinion that it should be as easy as possible to create something that looks relational. I assume option 3 would me much more work than option 2.
Thanks for the additional detail.
~jd
----- "Steven Hawkins" <shawkins(a)redhat.com> wrote:
> Option 1 : don't leave the xml world
>
> i.e. extract(xml, '/po', '/item/quantity/text() > 10',
> '/number/text()' as Number integer)
>
> Here the second argument specifies the nodes to consider, the third
> argument is the node test, and the remainder perform the typed scalar
> extractions. That's probably not very satisfactory for ad hoc
> querying, but it's very clear how it operates and what it should
> return.
>
> Option 2 : emulate the xml relational logic of adding synthetic
> primary keys
>
> i.e. select PO.Number from (extract(xml, '/po', 'position()' as key,
> '/number/text()' as Number integer)) AS PO,
> (extract(xml, '/po/item', '../position()' as ParentKey,
> '/quantity/text()' as Number integer)) AS Item where PO.key =
> Item.ParentKey and Item.Quantity > 10
>
> I'm omitting the node test parameter shown in option 1. The downside
> here is that you will perform multiple extractions and then join the
> results rather than option 1, which would perform everything in one
> step. This form does however make it a bit simpler to create
> procedures and the corresponding proc relational queries that map to
> something that looks relational. For example there would be a PO
> stored procedure that takes a doc as a parameter and returns the PO
> rowset, the procedure logic would be nothing more than extract(xmlin,
> '/po', 'position()' as key, '/number/text()' as Number integer);.
> There would also be one for Item. Then I could write a query like:
>
> select PO.Number from PO, Item where PO.xmlin = xml and Item.xmlin =
> xml and PO.key = Item.ParentKey and Item.Quantity > 10
>
> Option 3 : natively understand hierarchical xml structures similar to
> oql/hsql and our document model queries, in which the join between
> parent and child is implicit.
>
> i.e. Select PO.Number from xml where PO.Item.Quantity > 10
>
> however our xml document model query syntax in this case would project
> a document that only contains the Number element, not just the scalar
> number value in a rowset. We would also have to know the document
> model / schema used in the from clause - not just that it's typed as
> XML.
>
> ----- Original Message -----
> From: "John Doyle" <jdoyle(a)redhat.com>
> To: "Ramesh Reddy" <rareddy(a)redhat.com>
> Cc: "teiid-dev" <teiid-dev(a)lists.jboss.org>
> Sent: Thursday, September 17, 2009 1:49:43 PM GMT -06:00 US/Canada
> Central
> Subject: Re: [teiid-dev] XML as a source
>
> I'm not really arguing against XPath as the syntax to express the
> mapping, it's a good solution. I'm describing a higher level problem
> that I don't see the solution for.
>
> Lets say we have a virtual view we have created for a purchase order
> document and we have defined three tables with the proposed new
> functions: PO, Address, and Item. POs have both BillTo and ShipTo
> Addresses as well as a list of Items. Now we query this view with the
> following:
>
> Select PO.Number from PO, Item where Item.Quantity > 10;
>
> If our virtual tables defined by our functions are just returning row
> sets, how does this join get resolved? How does the engine know? In
> the XML-Relational implementation we create keyed relations in the
> model and then insert keys into the document as it is parsed so that
> we engine can resolve the join across the result sets we produce.
>
> ~jd
>
> ----- "Ramesh Reddy" <rareddy(a)redhat.com> wrote:
>
> > I see that in the current source model's table and column property
> > "NameInSource" already contains a xpath expression to get to column
> > information from a given XML document. How is proposed function
> based
> > model is different, than just making a what is source model now to
> a
> > virtual model in future. However, this forces the XML --> Table
> > mappings
> > will be done at the engine level, thus pushes relational issues to
> > engine. Am I interpreting the scenario correctly?
> >
> > On Thu, 2009-09-17 at 12:05 -0400, John Doyle wrote:
> > > I wanted to pick up the discussion started in TEIID_817 around
> > improvements/alternatives to the current implementation of the XML
> > Connectors (https://jira.jboss.org/jira/browse/TEIID-817).
> > Specifically I want to get a better understanding of the idea Steve
> > presented.
> > > ___
> > > ...
> > > The alternative we've talked about before is to treat the source
> > access and table extraction as primary and separate functions of
> > Teiid. Executing a web service could have a procedure like xml
> > getDoc(xml param) - which is essentially the approach of the XML
> > Source logic. On top of that there can be helper SQL/XML like
> > functions to map a rowset to xml, such as xmlforest, and the result
> > xml to "tables" - something like extract(xml param, xpath as col1,
> > xpath as col2....). At the very least adding a rowset or table
> type.
> > > __
> > >
> > >
> > > How do you envision exposing the functions, specifically
> functions
> > that turn XML into a rowset or table, to users? Would they be
> > available in Transformations that define virtual tables? This
> would
> > retain the central value of the XML-Relational connectors, and
> remove
> > much of the complexity of how it does it.
> > >
> > > One of the challenges in this is that we need to be able to
> extract
> > multiple virtual tables from a single XML document, and retain
> > (create) the relationships between them. For instance, if we
> extract
> > PO, Address and Item tables from a Purchase Order xml document, we
> > need to be able to map the item to the purchase order that it came
> > from. We do this currently by adding id's to the xml document and
> > exposing them in the model we generate.
> > >
> > > One thing I'll note is that passing around XML has it's limits.
> > Some of our customers are handling very large XML.
> > >
> > > ~jd
> >
> > _______________________________________________
> > teiid-dev mailing list
> > teiid-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/teiid-dev
> _______________________________________________
> teiid-dev mailing list
> teiid-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
16 years
6.3 planning
by Steven Hawkins
There hasn't been much on the dev or user list about work for 6.3, but it's time again to start being more formal about our expectations for the next release cycle. There are three main engineering goals for the 6.3 release cycle (~12 weeks):
1. To refactor caching/simple materialization.
This entails removing source result set caching (TEIID-168), retargeting user result set caching to use BufferManagement (this removes the memory-only restriction, and is more efficient since we already have the result captured in a tuplesource), to automatically scope cache results based upon the determinism level of the command, to create implicitly and explicit (through metadata or language directives) relation caches (which would be basically equivalent to temp tables) and to update planning to take advantage of the cached tables. Much of this is already discussed in TEIID-168, but needs expanded upon. The usage of Infinispan should be preferred as a replacement for our usage of JBoss cache given its touted performance improvements and better messaging of Teiid as "cloud ready". Infinispan could also be targeted as a backend for the BufferManager/temp tables (there would be an opportunity to address the temp table defects TEIID-189 TEIID-89 TEIID-162). And last, but not least, code tables should be reworked (TEIID-829)
2. Provide a deeply integrated container deployment mode.
This entails changes to our transaction handling and threading to allow for usage of a container supplied transaction manager (Needs a JIRA). Beyond that, Ramesh is investigating integrating Teiid as a JCA component - which can handle both embedded and server mode transaction/threading/security and JON discovery. The JBoss JCA project provides a simplified MC based deployment environment, that would serve as the basis for a replacement of standalone server deployment. Taken to a logical extreme we would also want to abstract the transport handling out of the XML/Salesforce connectors (Needs a JIRA) and rely on more container provided mechanisms.
3. Add a kit download for a "usable" JOPR console - Needs a JIRA, and continue ensuring adminapi/admin shell coverage of all exposed functionality.
As with 6.2, this work should have minimal impact on Designer's usage of Teiid embedded. Any new work in Designer would be a stretch goal only.
The engineering stretch goals for this release are:
1. Have definitive plans around Teiid owned metadata in a persistent form, which has already been on introduced as a topic on the dev-list.
2. Re-implement file and database based artifact access through JCR (DNA) - with an eye toward using JCR as a means to provide "cluster" features, but that would still be api compatible with the embedded implementation (JCR backed by the filesystem).
3. Allow for pass-through ddl in both the single source and multi-source scenarios - TEIID-669
4. Name-spaced/connector scoped functions to allow for broader function push-down. This could also be tied to supporting import and exposing JDBC 4.0 style function metadata (Needs a JIRA) See also (TEIID-502 TEIID-231)
5. Runtime costing update (TEIID-245)
6. Sub-query optimizations (TEIID-211)
... and many more that should be upped in priority ...
We should keep in minde that a 6.4 bucket may be needed depending upon the intermediate head way made on 7.x
The remaining 6.2 doc issues around our legacy SOAP services and some proposed REST services in terms of how they can be implemented with JBoss projects with a loose coupling to Teiid have been moved to 6.3.
On the non-engineering side of things we have the requirements of:
1. Updating the server extension guide to cover things from an embedded/simplified perspective given the changes to command logging, membership domains, etc. (slipped from 6.2)
2. Add the jdbc developers guide - TEIID-315 (slipped from 6.2)
The on-going ease of use issues are:
1. Adding additional screen casts covering a broader set of common uses.
2. Add more examples to the kit, especially highlighting use of non-relational sources.
3. Get Teiid as a recognized source in other products/projects - squirrel, dtp, hibernate, etc.
Please provide feedback on missing items or those that need more detail.
Note there are missing JIRAs for a lot of these items and the priority may not yet be assigned correctly for items identified as requirements. Devs and leads should feel free to help fill in gaps. Let me know if there is anything that has been left out or what we should dive into a little more detail on.
The Teiid Team
16 years
Re: [teiid-dev] XML as a source
by John Doyle
Option 2 seems like an attractive solution, I'm of the opinion that it should be as easy as possible to create something that looks relational. I assume option 3 would me much more work than option 2.
Thanks for the additional detail.
~jd
----- "Steven Hawkins" <shawkins(a)redhat.com> wrote:
> Option 1 : don't leave the xml world
>
> i.e. extract(xml, '/po', '/item/quantity/text() > 10',
> '/number/text()' as Number integer)
>
> Here the second argument specifies the nodes to consider, the third
> argument is the node test, and the remainder perform the typed scalar
> extractions. That's probably not very satisfactory for ad hoc
> querying, but it's very clear how it operates and what it should
> return.
>
> Option 2 : emulate the xml relational logic of adding synthetic
> primary keys
>
> i.e. select PO.Number from (extract(xml, '/po', 'position()' as key,
> '/number/text()' as Number integer)) AS PO,
> (extract(xml, '/po/item', '../position()' as ParentKey,
> '/quantity/text()' as Number integer)) AS Item where PO.key =
> Item.ParentKey and Item.Quantity > 10
>
> I'm omitting the node test parameter shown in option 1. The downside
> here is that you will perform multiple extractions and then join the
> results rather than option 1, which would perform everything in one
> step. This form does however make it a bit simpler to create
> procedures and the corresponding proc relational queries that map to
> something that looks relational. For example there would be a PO
> stored procedure that takes a doc as a parameter and returns the PO
> rowset, the procedure logic would be nothing more than extract(xmlin,
> '/po', 'position()' as key, '/number/text()' as Number integer);.
> There would also be one for Item. Then I could write a query like:
>
> select PO.Number from PO, Item where PO.xmlin = xml and Item.xmlin =
> xml and PO.key = Item.ParentKey and Item.Quantity > 10
>
> Option 3 : natively understand hierarchical xml structures similar to
> oql/hsql and our document model queries, in which the join between
> parent and child is implicit.
>
> i.e. Select PO.Number from xml where PO.Item.Quantity > 10
>
> however our xml document model query syntax in this case would project
> a document that only contains the Number element, not just the scalar
> number value in a rowset. We would also have to know the document
> model / schema used in the from clause - not just that it's typed as
> XML.
>
> ----- Original Message -----
> From: "John Doyle" <jdoyle(a)redhat.com>
> To: "Ramesh Reddy" <rareddy(a)redhat.com>
> Cc: "teiid-dev" <teiid-dev(a)lists.jboss.org>
> Sent: Thursday, September 17, 2009 1:49:43 PM GMT -06:00 US/Canada
> Central
> Subject: Re: [teiid-dev] XML as a source
>
> I'm not really arguing against XPath as the syntax to express the
> mapping, it's a good solution. I'm describing a higher level problem
> that I don't see the solution for.
>
> Lets say we have a virtual view we have created for a purchase order
> document and we have defined three tables with the proposed new
> functions: PO, Address, and Item. POs have both BillTo and ShipTo
> Addresses as well as a list of Items. Now we query this view with the
> following:
>
> Select PO.Number from PO, Item where Item.Quantity > 10;
>
> If our virtual tables defined by our functions are just returning row
> sets, how does this join get resolved? How does the engine know? In
> the XML-Relational implementation we create keyed relations in the
> model and then insert keys into the document as it is parsed so that
> we engine can resolve the join across the result sets we produce.
>
> ~jd
>
> ----- "Ramesh Reddy" <rareddy(a)redhat.com> wrote:
>
> > I see that in the current source model's table and column property
> > "NameInSource" already contains a xpath expression to get to column
> > information from a given XML document. How is proposed function
> based
> > model is different, than just making a what is source model now to
> a
> > virtual model in future. However, this forces the XML --> Table
> > mappings
> > will be done at the engine level, thus pushes relational issues to
> > engine. Am I interpreting the scenario correctly?
> >
> > On Thu, 2009-09-17 at 12:05 -0400, John Doyle wrote:
> > > I wanted to pick up the discussion started in TEIID_817 around
> > improvements/alternatives to the current implementation of the XML
> > Connectors (https://jira.jboss.org/jira/browse/TEIID-817).
> > Specifically I want to get a better understanding of the idea Steve
> > presented.
> > > ___
> > > ...
> > > The alternative we've talked about before is to treat the source
> > access and table extraction as primary and separate functions of
> > Teiid. Executing a web service could have a procedure like xml
> > getDoc(xml param) - which is essentially the approach of the XML
> > Source logic. On top of that there can be helper SQL/XML like
> > functions to map a rowset to xml, such as xmlforest, and the result
> > xml to "tables" - something like extract(xml param, xpath as col1,
> > xpath as col2....). At the very least adding a rowset or table
> type.
> > > __
> > >
> > >
> > > How do you envision exposing the functions, specifically
> functions
> > that turn XML into a rowset or table, to users? Would they be
> > available in Transformations that define virtual tables? This
> would
> > retain the central value of the XML-Relational connectors, and
> remove
> > much of the complexity of how it does it.
> > >
> > > One of the challenges in this is that we need to be able to
> extract
> > multiple virtual tables from a single XML document, and retain
> > (create) the relationships between them. For instance, if we
> extract
> > PO, Address and Item tables from a Purchase Order xml document, we
> > need to be able to map the item to the purchase order that it came
> > from. We do this currently by adding id's to the xml document and
> > exposing them in the model we generate.
> > >
> > > One thing I'll note is that passing around XML has it's limits.
> > Some of our customers are handling very large XML.
> > >
> > > ~jd
> >
> > _______________________________________________
> > teiid-dev mailing list
> > teiid-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/teiid-dev
> _______________________________________________
> teiid-dev mailing list
> teiid-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
16 years
Re: [teiid-dev] XML as a source
by Steven Hawkins
Option 1 : don't leave the xml world
i.e. extract(xml, '/po', '/item/quantity/text() > 10', '/number/text()' as Number integer)
Here the second argument specifies the nodes to consider, the third argument is the node test, and the remainder perform the typed scalar extractions. That's probably not very satisfactory for ad hoc querying, but it's very clear how it operates and what it should return.
Option 2 : emulate the xml relational logic of adding synthetic primary keys
i.e. select PO.Number from (extract(xml, '/po', 'position()' as key, '/number/text()' as Number integer)) AS PO,
(extract(xml, '/po/item', '../position()' as ParentKey, '/quantity/text()' as Number integer)) AS Item where PO.key = Item.ParentKey and Item.Quantity > 10
I'm omitting the node test parameter shown in option 1. The downside here is that you will perform multiple extractions and then join the results rather than option 1, which would perform everything in one step. This form does however make it a bit simpler to create procedures and the corresponding proc relational queries that map to something that looks relational. For example there would be a PO stored procedure that takes a doc as a parameter and returns the PO rowset, the procedure logic would be nothing more than extract(xmlin, '/po', 'position()' as key, '/number/text()' as Number integer);. There would also be one for Item. Then I could write a query like:
select PO.Number from PO, Item where PO.xmlin = xml and Item.xmlin = xml and PO.key = Item.ParentKey and Item.Quantity > 10
Option 3 : natively understand hierarchical xml structures similar to oql/hsql and our document model queries, in which the join between parent and child is implicit.
i.e. Select PO.Number from xml where PO.Item.Quantity > 10
however our xml document model query syntax in this case would project a document that only contains the Number element, not just the scalar number value in a rowset. We would also have to know the document model / schema used in the from clause - not just that it's typed as XML.
----- Original Message -----
From: "John Doyle" <jdoyle(a)redhat.com>
To: "Ramesh Reddy" <rareddy(a)redhat.com>
Cc: "teiid-dev" <teiid-dev(a)lists.jboss.org>
Sent: Thursday, September 17, 2009 1:49:43 PM GMT -06:00 US/Canada Central
Subject: Re: [teiid-dev] XML as a source
I'm not really arguing against XPath as the syntax to express the mapping, it's a good solution. I'm describing a higher level problem that I don't see the solution for.
Lets say we have a virtual view we have created for a purchase order document and we have defined three tables with the proposed new functions: PO, Address, and Item. POs have both BillTo and ShipTo Addresses as well as a list of Items. Now we query this view with the following:
Select PO.Number from PO, Item where Item.Quantity > 10;
If our virtual tables defined by our functions are just returning row sets, how does this join get resolved? How does the engine know? In the XML-Relational implementation we create keyed relations in the model and then insert keys into the document as it is parsed so that we engine can resolve the join across the result sets we produce.
~jd
----- "Ramesh Reddy" <rareddy(a)redhat.com> wrote:
> I see that in the current source model's table and column property
> "NameInSource" already contains a xpath expression to get to column
> information from a given XML document. How is proposed function based
> model is different, than just making a what is source model now to a
> virtual model in future. However, this forces the XML --> Table
> mappings
> will be done at the engine level, thus pushes relational issues to
> engine. Am I interpreting the scenario correctly?
>
> On Thu, 2009-09-17 at 12:05 -0400, John Doyle wrote:
> > I wanted to pick up the discussion started in TEIID_817 around
> improvements/alternatives to the current implementation of the XML
> Connectors (https://jira.jboss.org/jira/browse/TEIID-817).
> Specifically I want to get a better understanding of the idea Steve
> presented.
> > ___
> > ...
> > The alternative we've talked about before is to treat the source
> access and table extraction as primary and separate functions of
> Teiid. Executing a web service could have a procedure like xml
> getDoc(xml param) - which is essentially the approach of the XML
> Source logic. On top of that there can be helper SQL/XML like
> functions to map a rowset to xml, such as xmlforest, and the result
> xml to "tables" - something like extract(xml param, xpath as col1,
> xpath as col2....). At the very least adding a rowset or table type.
> > __
> >
> >
> > How do you envision exposing the functions, specifically functions
> that turn XML into a rowset or table, to users? Would they be
> available in Transformations that define virtual tables? This would
> retain the central value of the XML-Relational connectors, and remove
> much of the complexity of how it does it.
> >
> > One of the challenges in this is that we need to be able to extract
> multiple virtual tables from a single XML document, and retain
> (create) the relationships between them. For instance, if we extract
> PO, Address and Item tables from a Purchase Order xml document, we
> need to be able to map the item to the purchase order that it came
> from. We do this currently by adding id's to the xml document and
> exposing them in the model we generate.
> >
> > One thing I'll note is that passing around XML has it's limits.
> Some of our customers are handling very large XML.
> >
> > ~jd
>
> _______________________________________________
> teiid-dev mailing list
> teiid-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
_______________________________________________
teiid-dev mailing list
teiid-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/teiid-dev
16 years
Re: [teiid-dev] XML as a source
by John Doyle
That SQL doesn't make any sense you say. Right you are. :) But it does get to my point.
How do we create SQL for this:
Get all PO.Numbers where any of the items has a quantity greater than 10.
~jd
----- "John Doyle" <jdoyle(a)redhat.com> wrote:
> I'm not really arguing against XPath as the syntax to express the
> mapping, it's a good solution. I'm describing a higher level problem
> that I don't see the solution for.
>
> Lets say we have a virtual view we have created for a purchase order
> document and we have defined three tables with the proposed new
> functions: PO, Address, and Item. POs have both BillTo and ShipTo
> Addresses as well as a list of Items. Now we query this view with the
> following:
>
> Select PO.Number from PO, Item where Item.Quantity > 10;
>
> If our virtual tables defined by our functions are just returning row
> sets, how does this join get resolved? How does the engine know? In
> the XML-Relational implementation we create keyed relations in the
> model and then insert keys into the document as it is parsed so that
> we engine can resolve the join across the result sets we produce.
>
> ~jd
>
> ----- "Ramesh Reddy" <rareddy(a)redhat.com> wrote:
>
> > I see that in the current source model's table and column property
> > "NameInSource" already contains a xpath expression to get to column
> > information from a given XML document. How is proposed function
> based
> > model is different, than just making a what is source model now to
> a
> > virtual model in future. However, this forces the XML --> Table
> > mappings
> > will be done at the engine level, thus pushes relational issues to
> > engine. Am I interpreting the scenario correctly?
> >
> > On Thu, 2009-09-17 at 12:05 -0400, John Doyle wrote:
> > > I wanted to pick up the discussion started in TEIID_817 around
> > improvements/alternatives to the current implementation of the XML
> > Connectors (https://jira.jboss.org/jira/browse/TEIID-817).
> > Specifically I want to get a better understanding of the idea Steve
> > presented.
> > > ___
> > > ...
> > > The alternative we've talked about before is to treat the source
> > access and table extraction as primary and separate functions of
> > Teiid. Executing a web service could have a procedure like xml
> > getDoc(xml param) - which is essentially the approach of the XML
> > Source logic. On top of that there can be helper SQL/XML like
> > functions to map a rowset to xml, such as xmlforest, and the result
> > xml to "tables" - something like extract(xml param, xpath as col1,
> > xpath as col2....). At the very least adding a rowset or table
> type.
> > > __
> > >
> > >
> > > How do you envision exposing the functions, specifically
> functions
> > that turn XML into a rowset or table, to users? Would they be
> > available in Transformations that define virtual tables? This
> would
> > retain the central value of the XML-Relational connectors, and
> remove
> > much of the complexity of how it does it.
> > >
> > > One of the challenges in this is that we need to be able to
> extract
> > multiple virtual tables from a single XML document, and retain
> > (create) the relationships between them. For instance, if we
> extract
> > PO, Address and Item tables from a Purchase Order xml document, we
> > need to be able to map the item to the purchase order that it came
> > from. We do this currently by adding id's to the xml document and
> > exposing them in the model we generate.
> > >
> > > One thing I'll note is that passing around XML has it's limits.
> > Some of our customers are handling very large XML.
> > >
> > > ~jd
> >
> > _______________________________________________
> > teiid-dev mailing list
> > teiid-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/teiid-dev
> _______________________________________________
> teiid-dev mailing list
> teiid-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
16 years
Re: [teiid-dev] XML as a source
by John Doyle
I'm not really arguing against XPath as the syntax to express the mapping, it's a good solution. I'm describing a higher level problem that I don't see the solution for.
Lets say we have a virtual view we have created for a purchase order document and we have defined three tables with the proposed new functions: PO, Address, and Item. POs have both BillTo and ShipTo Addresses as well as a list of Items. Now we query this view with the following:
Select PO.Number from PO, Item where Item.Quantity > 10;
If our virtual tables defined by our functions are just returning row sets, how does this join get resolved? How does the engine know? In the XML-Relational implementation we create keyed relations in the model and then insert keys into the document as it is parsed so that we engine can resolve the join across the result sets we produce.
~jd
----- "Ramesh Reddy" <rareddy(a)redhat.com> wrote:
> I see that in the current source model's table and column property
> "NameInSource" already contains a xpath expression to get to column
> information from a given XML document. How is proposed function based
> model is different, than just making a what is source model now to a
> virtual model in future. However, this forces the XML --> Table
> mappings
> will be done at the engine level, thus pushes relational issues to
> engine. Am I interpreting the scenario correctly?
>
> On Thu, 2009-09-17 at 12:05 -0400, John Doyle wrote:
> > I wanted to pick up the discussion started in TEIID_817 around
> improvements/alternatives to the current implementation of the XML
> Connectors (https://jira.jboss.org/jira/browse/TEIID-817).
> Specifically I want to get a better understanding of the idea Steve
> presented.
> > ___
> > ...
> > The alternative we've talked about before is to treat the source
> access and table extraction as primary and separate functions of
> Teiid. Executing a web service could have a procedure like xml
> getDoc(xml param) - which is essentially the approach of the XML
> Source logic. On top of that there can be helper SQL/XML like
> functions to map a rowset to xml, such as xmlforest, and the result
> xml to "tables" - something like extract(xml param, xpath as col1,
> xpath as col2....). At the very least adding a rowset or table type.
> > __
> >
> >
> > How do you envision exposing the functions, specifically functions
> that turn XML into a rowset or table, to users? Would they be
> available in Transformations that define virtual tables? This would
> retain the central value of the XML-Relational connectors, and remove
> much of the complexity of how it does it.
> >
> > One of the challenges in this is that we need to be able to extract
> multiple virtual tables from a single XML document, and retain
> (create) the relationships between them. For instance, if we extract
> PO, Address and Item tables from a Purchase Order xml document, we
> need to be able to map the item to the purchase order that it came
> from. We do this currently by adding id's to the xml document and
> exposing them in the model we generate.
> >
> > One thing I'll note is that passing around XML has it's limits.
> Some of our customers are handling very large XML.
> >
> > ~jd
>
> _______________________________________________
> teiid-dev mailing list
> teiid-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/teiid-dev
16 years
XML as a source
by John Doyle
I wanted to pick up the discussion started in TEIID_817 around improvements/alternatives to the current implementation of the XML Connectors (https://jira.jboss.org/jira/browse/TEIID-817). Specifically I want to get a better understanding of the idea Steve presented.
___
...
The alternative we've talked about before is to treat the source access and table extraction as primary and separate functions of Teiid. Executing a web service could have a procedure like xml getDoc(xml param) - which is essentially the approach of the XML Source logic. On top of that there can be helper SQL/XML like functions to map a rowset to xml, such as xmlforest, and the result xml to "tables" - something like extract(xml param, xpath as col1, xpath as col2....). At the very least adding a rowset or table type.
__
How do you envision exposing the functions, specifically functions that turn XML into a rowset or table, to users? Would they be available in Transformations that define virtual tables? This would retain the central value of the XML-Relational connectors, and remove much of the complexity of how it does it.
One of the challenges in this is that we need to be able to extract multiple virtual tables from a single XML document, and retain (create) the relationships between them. For instance, if we extract PO, Address and Item tables from a Purchase Order xml document, we need to be able to map the item to the purchase order that it came from. We do this currently by adding id's to the xml document and exposing them in the model we generate.
One thing I'll note is that passing around XML has it's limits. Some of our customers are handling very large XML.
~jd
16 years
Joined the group
by Sanjay Chaudhuri
Hi Ramesh,
I have joined this group. Trying to select a project as a contributor, I
did not see any project listed as CDK Toolkit; all I found was Teiid and
Teiid Designer. Do I register under Teiid ?
Thanks
Sanjay
16 years
6.2 RC1 is now available
by Steven Hawkins
Hello all,
Have a look at the latest RC1 kit. All has been going well with Designer integration so far, so the release may be a soon as the end of the week.
The Teiid Team
16 years
