[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2253: --------------------------------------- The common workaround for all of these issues is to use a partitioned union - see https://docs.jboss.org/author/display/TEIID/Federated+Optimizations. The intent of this issue is to update and rationalize the multi-source feature. Even if the sub-tasks are addressed a core limitation of multi-source would be the requirement of a homogenous schema (all tables/columns must share the same name) and for backwards compatibility joins must implicitly specify tbl1.source_name = tb2.source_name - to enforce that the partitioned join is used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2253: ------------------------------------- I do not understand why this will preclude the dynamic, can you pls explain? Teiid always cached the metadata on the deploy, I am not sure how this is different from metadata capture perspective, other than collecting from all the sources and providing a view on top of all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2253: ------------------------------------- sounds good, my only concern is metadata size, especially when we have one of those 800+ sources. May be we limit the merge to unique source type by the translator name, or a configuration property? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2253: --------------------------------------- I've been spending some time with trying to make the multi-source logic more consistent as part of this overhaul. It looks like we'll end up with a very different implementation than we do now. Of current logic is based upon the idea that the plans will be mostly pushdown/simple such that the effect of the multi-source operation can be localized to the access nodes. In general this is not the case, for example if multi-source tables end up on both sides of a join, in a with clause, on both sides of an insert into, etc. the current logic does not correctly interpret the intent. The approach of disabling all ordering/aggregate capabilities is also not ideal as it should only affect the user query. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2253: --------------------------------------- I don't follow you. They are separate sources regardless of whether a multi-source feature is used or not. Do you mean each source having it's own metadata/schema? That is a possible eventual direction. My near term (8.3) goal is to just update what we have to remove the limitations highlighted in this issue while still providing a mostly, if not, non-breaking feature compared to the existing multi-source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2253: ------------------------------------- What I mean is when they are on both sides of the join, why not treat treat them non-multi source query? I was asking more generically how this separate multi-source processing helping in reducing any complexity? One thing I do understand it creates that pseudo virtual model that has all the sources in union, makes the job lot easier for the user to not write this layer explicitly. That is good enough reason, as I think again -:) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/TEIID-2253?page=com.atlassian.jira.plugin... ] Steven Hawkins resolved TEIID-2253. ----------------------------------- Fix Version/s: 8.3 Resolution: Done Marking the parent issue as resolved, as we will not look to address TEIID-2251 anytime soon. Thus TEIID-2251 will serve as a KI and discussion focus. The changes greatly rationalized the multi-source feature. It is now also a primary concern of the engine and thus optimization aware rather than being a simplistic override. If the org.teiid.implicitMultiSourceJoin is set to false, then feature forces the user to specify explicit join criteria, which ensures the results will be as expected. The old logic (and the current logic with implicit joins) suffers from not properly partitioning results if a join between to multi-source tables from the same model is not pushed down - which can happen for a variety of reasons (for example unsupported join criteria). The possibly breaking changes here are: 1. The multi-source column is now a pseudo column and is not selectable in wildcards. This seems to be an oversight of the old feature since we lacked the differentiation between a pseudo column and something that was non-selectable (which would have prevented it's use at all in projection). 2. Multi-source inserts cannot be broadcast to all sources, rather they must target a single source. This is consistent with the partitioned view logic and logically follows since multi-source is functioning on a sharding not replication premise. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira